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TNTRODUCTTON 


The  Spoken  Human-Machine  Dialogue  Workshop  sponsored  by  the  Army  Research  Office  (ARO), 
with  participation  by  TRADOC,  was  held  at  the  Sheraton  Imperial  Hotel  on  May  30  through  June 
1, 1995.  This  workshop  was  timely  because  university  research  in  natural-language  processing  has 
reached  a  level  of  maturity  appropriate  for  examining  the  integration  of  natural-language  systems  into 
requirements  and  prototypes  that  support  the  realization  of  Army  Force  XXI  capabilities.  There  also 
is  a  current  need  to  identify  and  focus  future  basic  research  to  ensure  that  the  full  potential  of  natural- 
language  processing  is  realized  for  both  military  and  civilian  applications. 

Attendance  at  the  workshop  was  by  invitation  only.  Attachments  A,  B  and  C  are  respectively;  the 
letter  of  invitation;  administrative  instructions  to  the  attendees;  and  lit  of  attendees.  The  workshop 
agenda  is  presented  in  Attachment  D. 

The  workshop  started  with  presentations  by  three  leading  researchers  in  natural-language  processing 
in  the  US.  Cliff  Weinstein  of  the  Massachusetts  Institute  of  Technology  presented  his  view  of  spoken 
human-machine  dialogue  25  years  in  the  future  based  on  progress  achieved  over  the  past  25  years. 
Mitch  Marcus  from  the  University  of  Pennsylvania  presented  current  research  and  development 
efforts  in  natural-language  processing  and  its  implications  for  applications  now  and  in  the  future. 
Alan  Biermann,  Duke  University,  discussed  natural-language  processing  concepts  within  the 
framework  of  spoken  human-machine  dialogue  systems. 

Following  the  three  introductory  presentations,  Jim  Brown,  Curry  Guinn  and  Hal  Waters—all  of 
Research  Triangle  Institute-discussed  and  demonstrated  training  and  intelligent  assistant  systems 
incorporating  virtual  reality  and  spoken  human-machine  dialogue  technologies.  Demonstrations 
concluded  with  a  discussion  of  integrating  both  spoken  dialogue  and  virtual  reality  in  such  systems. 

The  after-dirmer  speaker,  Thomas  Edwards,  made  an  interesting  and  impressive  presentation  that 
graphically  depicted  the  application  of  advanced  communications  technology  under  battle  conditions. 

Each  participant  was  assigned  to  one  of  sbc  Working  Groups  during  breakout  sessions  to  examine  and 
develop  issues  and  applications  of  spoken  human-machine  dialogue.  Each  Working  Group  was 
chaired  by  a  representative  for  the  TRADOC  BattleLabs  and  co-chaired  by  a  representative  of  the 
Army's  scientific  community.  Where  possible  participants  were  given  their  preferences  (1-7)  for 
assignment  to  the  Working  Group. 

Each  working  group  met  for  a  total  of  6  hours  to  consider  integration  of  natural-language  systems 
into  requirements  and  prototypes  that  support  the  realization  of  Army  Force  XXI  capabilities  and  to 
identify  future  basic  research  in  three  applications  areas:  training;  assistance;  and  operational. 

The  working  groups  and  their  co-chairs  were: 

•  Battle  Command 

Co-Chairs:  CPT  Kelly  Lauritzen,  Cliff  Weinstein 
1.  _ 

1  Original  plans  included  a  Dismounted  Warfighting  working  group.  This  was  omitted  at  the  workshop  because  appropriate 
US  Army  representatives  were  not  able  to  attend. 


1 


•  Mounted  Warfighting  Battlespace 

Co-Chairs:  Chuck  Campbell,  Curry  Guinn 

•  Combat  Service  Support 

Co-Chairs:  Howard  Burnette,  A.  Joshi 

•  Depth  and  Simultaneous  Attack 

Co-Chairs:  Donald  Starch,  Don  Per  Us 

•  Earl  Entry  Lethality  and  Survivability 

Co-Chairs:  Dave  Carter,  Alan  Biermann 

•  Louisiana  Maneuvers  Task  Force 

Co-Chairs:  Mitch  Marcus,  Deanna  Jenkins 

On  the  third  day  of  the  workshop,  the  co-chairs  of  the  working  groups  presented  their  findings. 
These  findings  are  elaborated  in  the  body  of  this  report.  Finally,  MAJ  Brad  Shaffer  summarized  the 
findings  of  the  workshop  and  closing  remarks  were  made  by  J.  Chandra,  ARO. 
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SPOKEN  LANGUAGE  DIALOGUE 

Alan  W.  Biermann 
Department  of  Computer  Science 
Duke  University 

The  purpose  of  dialogue  theory  is  to  enable  fast  convergence  to  a  goal  in  a  cooperative  effort 
between  interacting  agents.  The  methods  are  toimplement  the  discovery  of  critical  factors  related  to 
the  achievement  of  success,  the  immediate  focus  of  the  conversational  partners  efforts  on  those 
factors,  and  the  proper  assigning  of  duties  according  to  their  respective  capabilites.  The  model  for 
the  target  behaviors  is  that  observed  as  human  collaborators  attempt  to  solve  a  problem  in  a  voice 
interactive  environment.  They  usually  can  quickly  perceive  the  essential  bottlenecks  to  the  solution 
of  the  problem  and  they  concentrate  efforts  on  those.  Their  conversation  will  tend  to  jump  quickly 
from  topic  to  topic  as  the  needs  of  the  task  point  to  new  issues  to  be  adddressed.  All  interactions 
account  for  the  abilities  of  each  participant  with  appropriate  voice  interactions  that  omit  redundancy 
and  explain  only  necessary  issues. 

Such  behaviors  are  in  contrast  to  typical  interactions  with  computers  which  tend  to  involve  only  linear 
flow  of  control,  a  total  inability  to  jump  to  key  issues  as  the  task  requires,  and  little  adaptation  to  the 
user. 

We  present  a  theory  of  dialogue  that  is  based  upon  Prolog-like  theorem  proving  and  that  is  aimed  at 
enabling  a  machine  to  efficiently  interact  with  a  human  in  a  problem  solving  situation.  The 
mechanisms  attempt  to  prove  that  the  conversational  goal  has  been  acheived  and  then  they  isolate  key 
parts  of  the  proof  that  are  not  present  but  which  could  possibly  be  satisfied  through  interaction  with 
the  human  collaborator.  Then  the  system  interacts  with  the  user  to  try  to  achieve  those  needed 
subgoals  to  complete  the  proof  If  the  user  knows  a  method  to  solve  the  problem,  he  or  she  can  take 
control  of  the  interaction  and  lead  the  machine  to  a  solution.  The  system  can  easily  account  for  user 
knowledge  by  representing  it  as  a  part  of  the  Prolog  database  and  letting  the  natural  theorem  proving 
process  discover  the  appropriate  interactions.  Thus  the  system,  assuming  a  correctly  updated  user 
model,  will  not  waste  time  requesting  user  actions  that  are  not  feasible  or  by  over  explaining  issues 
that  the  user  already  understands. 

The  current  state  of  the  art  in  speech  recognition  and  dialogue  theory  seems  to  indicate  that  these 
technologies  are  ready  for  application  to  U.  S.  Army  applications.  Some  example  applications  would 
be  information  retrieval,  equipment  repair,  battle  management,  and  automatic  tutoring. 


Supported  by.  NSF  Grant  No.  IRI  9221842,  ONR  Grant  No.  NOOO 14-94- 1-093 8,  and  Research 
Triangle  Institute  subgrant  (from  ARO) 
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SPOKEN  LANGUAGE  DIAI  OGTTF. 


Mitch  Marcus 

Computer  Science  Department 
University  of  Pennsylvania 

This  report  summarizes  an  overview  of  the  state  of  the  art  in  automatic  speech  recognition  and 
natural  language  processing  by  computer.  It  focuses  on  two  topics:  the  capabilities  of  current  speech 
recognition  systems  and  a  recent  major  advance  in  natural  language  processing  which  is  already 
responsible  for  major  improvement  in  components  of  such  systems. 

A  Brief  Consumer’s  Guide  to  Speech  Recognition 

The  emergence  of  effective  speech  recognition  within  the  last  five  years  has  depended  on  finding 
methods  to  solve  several  simultaneous  problems. 

First,  the  segmentation  of  the  speech  stream  into  individual  words  and  letters  is  extremely  difficult. 
When  we  listen  to  speech,  we  often  have  the  very  strong  illusion  that  speech  consists  of  speech 
sounds  and  words  strung  together  like  beads  on  a  chain,  each  word  touching  but  distinct  from  the 
preceding  and  following  word.  The  acoustic  reality  is  very  different.  Examining  a  speech 
spectrogram,  a  visual  display  of  speech,  shows  that  there  is  no  noticeable  boundary  between  either 
speech  sounds  or  words  at  all.  There  are  occasional  pauses  at  the  ends  of  groups  of  words  while  the 
speaker  breathes,  and  there  are  very  brief  silences  produced  during  the  production  of  so-called  "stop" 
consonants,  P)t,k,  b,  d,  g,  when  the  tongue  or  lips  briefly  block  the  production  of  sound,  but  words 
otherwise  flow  seamlessly  into  each  other. 

Second,  speech  information  is  frequently  not  local.  For  example,  while  our  perception,  reflected  by 
spelling,  is  that  the  words  dose  and  doze  differ  in  the  final  consonant,  s  versus  z,  an  examination  of 
a  speech  spectrogram  shows  that  these  consonants  are  acoustically  identical.  The  difference  in  the 
identity  of  the  two  consonants  is  actually  encoded  in  the  length  of  the  preceding  vowel;  the  o  in  doze 
is  much  longer  than  the  o  in  dose. 

Third,  there  is  tremendous  variation  in  speech  production  both  across  talkers  and  within  the 
production  of  a  single  talker.  There  is  obvious  variation  between  speakers  due  to  differences  in 
gender,  dialect,  and  the  like,  but  even  the  same  word  said  by  the  same  speaker  can  vary  tremendously 
acoustically  depending  on  volume,  the  speed  of  the  speech,  and  voice  quality.  Whispered  speech,  for 
example,  is  fundamentally  different  acoustically  from  very  softly  spoken  normal  speech;  the  latter  is 
generated  by  a  very  soft  but  steady  periodic  tone  from  the  vocal  cords,  while  whispered  speech  is 
generated  by  the  aperiodic  noise  of  air  turbulence  at  the  back  of  the  throat. 

From  an  engineering  point  of  view,  the  difficult  of  a  speech  recognition  task  can  be  characterized 
along  a  variety  of  dimensions; 

(1)  Isolated  word  speech,  where  the  speaker  explicitly  stops  speaking  briefly  between  each  word, 
is  easier  to  recognize  than  continuous  speech. 
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(2)  Speaker  dependent  systems,  systems  which  are  explicitly  trained  for  each  new  talker,  perform 
better  than  systems  that  are  speaker  independent,  which  work  immediately  for  any  talker  with  no 
previous  training  for  that  talker's  voice.  A  good  middle  ground  is  provided  by  speaker  adaptive 
systems,  which  adapt  themselves  to  a  given  speaker  during  use,  rapidly  improving  in  performance 
from  some  original  baseline. 

(3)  Systems  with  small  vocabularies,  of  between  10  and  50  words,  have  much  better  performance 
than  systems  with  large  vocabularies,  of  1000  to  1000,000  words. 

(4)  Of  somewhat  more  importance  than  the  vocabulary  size  for  a  particular  task,  however,  is  a 
measure  called  task  perplexity.  Because  of  the  difficulty  of  the  recognition  task,  most  current 
systems  constrain  the  task  by  combining  a  fixed  vocabulary  with  some  form  of  probabilistic 
language  model  which  captures  the  expected  probabilities  of  particular  word  subsequences.  Such 
a  language  model  can  be  used  to  calculate  the  degree  of  ambiguity  at  each  point  of  the  parsing 
process;  perplexity  is  one  measure  of  this  ambiguity. 

(5)  Finally,  the  recording  conditions  make  a  large  difference  in  how  well  speech  recognition  systems 
work.  The  use  of  noise-cancelling  microphones  is  crucial  for  good  recognition  in  even  moderately 
noisy  environments;  high  noise  levels  greatly  degrade  the  accuracy  of  even  the  best  systems. 

Currently,  very  good  performance  can  be  achieved  by  constraining  the  recognition  task  along  one  or 
several  of  the  above  dimensions.  For  example,  AT\&T  Bell  Laboratories  has  achieved  recognition 
rates  of  99.2\%  string  accuracy  for  a  continuous  speech,  speaker  independent,  phone  quality  speech 
recognition  system  which  works  with  a  very  small  vocabulary  -  the  ten  digits,  if  the  string  length  is 
known.  Several  speaker  dependent  systems  with  word  error  rates  of  l-2\%  have  been  developed  for 
moderate  vocabulary  sizes  (1000-5000  words)  for  moderate  perplexity  (30-60).  In  one  application 
that  has  been  the  focus  of  several  years  of  international  competition,  namely  the  task  of  making 
reservations  for  airline  travel  given  the  information  in  the  Official  Airline  Guide,  systems  now 
correctly  understand  about  90\%  of  utterances  input  by  naive  users  during  evaluations  where  the  goal 
was  to  accomplish  some  particular  task. 

In  summary,  by  tightly  controlling  task  complexity  and  vocabulary  size,  very  good  results  can  be 
achieved  in  hands-busy,  eyes-busy  environments  using  current  speech  recognition  technologies. 

New  Developments  in  Natural  Language  Processing 

The  field  of  natural  language  processing  (NLP)  has  undergone  a  revolution  within  the  past  several 
years.  New  probabilistic  and  information-theoretic  techniques,  combined  with  the  traditional  methods 
of  symbolic  computing,  have  dramatically  improved  the  performance  of  a  variety  of  NLP  system 
components,  with  continuing  improvement  expected.  In  many  areas.  The  performance  of  techniques 
which  required  very  costly  hand-encoded  knowledge  representations  have  now  been  surpassed  by 
statistical  learning  methods  which  automatically  extract  information  from  large  bodies  of  annotated 
and  unannotated  online  text.  Much  of  this  work  has  been  done  with  ARO  and  other  DOD  funding, 
or  by  researchers  originally  trained  under  ARO  funding.  Important  breakthroughs  have  been  achieved 
in  many  key  areas;  we  will  focus  here  on  three  key  examples  to  give  a  sense  of  the  kind  of  techniques 
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that  have  recently  been  developed.  These  are: 

(1)  The  problem  of  part  of  speech  determination  --  deciding,  for  example,  whether  a  particular  word 
in  a  given  linguistic  context  functions  as  a  singular  noun  or  a  past  tense  verb.  A  variety  of  techniques 
now  tag  previously  unseen  material  at  about  96-97\%  word  accuracy. 

(2)  The  problem  of  determining  the  syntactic  structure  of  naturally  occurring  texts  and  utterances. 
Recently  developed  probabilistic  parsing  methods,  operating  completely  without  hand-built  grammars, 
provide  the  correct  parse  as  the  first  parse  output  between  60-80\%  of  the  time,  by  sentence,  on 
newspaper  text;  they  have  performed  with  up  to  91\%  accuracy  on  spoken  language  tasks. 

(3)  The  problem  of  determining  the  lexical  semantics  of  verbs.  For  example,  a  range  of  promising 
techniques  for  performing  automatic  word  sense  disambiguation  have  now  emerged,  i.e.  whether  a 
given  instance  of  plant  means  a  growing  thing  with  leaves  or  a  factory.  Most  surprisingly,  the  best 
technique  to  date  uses  an  tmsupervised  training  method. 

We  will  briefly  consider  each  in  turn. 

Part  of  Speech  Tagging 

To  determine  the  particular  grammatical  use  of  each  word  token  in  an  input  stream,  a  POS  tagger 
must  disambiguate  words  which  have  the  potential  to  be  many  different  parts  of  speech.  Overcoming 
this  combinatoric  bottleneck  is  a  central  problem  in  building  any  NLP  system;  given  a  realistically 
large  lexicon  of  English,  many  common  words  are  used  in  multiple  parts  of  speech.  Determining 
grammatical  function  is  a  crucial  step  toward  a  wide  range  of  tasks  ranging  from  information 
extraction  to  providing  a  chunking  of  text  into  intonation  phrases  in  a  text-to-speech  synthesizer.  The 
problem  of  lexical  disambiguation  was  widely  believed  to  be  completely  intractable  ten  years  ago,  and 
yet  now  a  wide  range  of  very  different  learning  techniques  can  solve  this  problem  with  between  95\% 
and  97.5\%  word  accuracy,  for  sets  of  between  40  and  80  grammatical  functions.  All  of  these 
techniques  depend,  however,  on  availability  of  hand-tagged  materials  for  training.  The  best  such 
results  have  been  obtained,  somewhat  surprisingly,  using  a  new  learning  method  which  results  in  a 
simple  list  of  symbolic  if-then  rules;  this  method  was  developed  by  Eric  Brill  at  UPenn,  in  part  under 
ARO  funding.  Very  recently.  Brill,  now  at  Johns  Hopkins,  has  extended  this  symbolic  learning 
method  to  produce  a  95%  correct  part-of-speech  tagger  for  English,  seeded  only  with  the  information 
in  a  standard  machine  readable  dictionary. 

Stochastic  Parsing 

Both  the  purely  symbolic  parsing  techniques  and  the  straightforward  combination  of  symbolic  parsing 
techniques  with  statistical  methods  that  was  explored  in  the  1980’s  failed  to  produce  effective  parsers; 
the  best  parsers  from  this  period  achieve  about  35\%  correct  parses  on  fully  reserved  test  wire-service 
material. 

Within  recent  years,  a  range  of  new  grammatical  formalisms  have  been  proposed  which  may  well  have 
the  potential  to  solve  a  major  part  of  this  problem.  These  formalisms,  called  lexicalized  grammar 
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formalisms,  express  grammars  in  which  the  entire  grammar  consists  of  complex  structures  associated 
with  individual  words,  plus  some  very  simple  general  rules  for  combining  these  structures.  In  these 
lexicalized  formalisms,  each  word  can  be  thought  of  as  a  tree  fragment;  the  full  grammatical  analysis 
of  a  sentence  is  formed  by  specifying  how  and  in  what  order  the  fragments  associated  with  each  word 
in  a  sentence  combine.  In  these  grammar  formalisms,  the  bulk  of  parsing  a  sentence  is  just  deciding 
on  which  "part  of  speech"  to  assign  to  each  word.  Work  at  the  University  of  Pennsylvania  in  viewing 
parsing  as  the  selection  of  these  "super  parts  of  speech"  has  recently  been  very  successful. 

One  new,  veiy  successful,  class  of  parsing  models  uses  conditional  probabilities  to  model  a  rich  range 
of  linguistic  structure.  The  first  successful  parser  of  this  kind,  developed  by  Magerman  and  myself, 
conditions  rule  probabilities  on  both  the  type  of  the  immediately  surrounding  linguistic  fragment,  and 
the  most  likely  part  of  speech  assignments  for  each  of  the  next  two  words  in  the  input.  This  parser 
correctly  parsed  89\%  of  a  reserved  test  set  from  a  simple  human-computer  interactive  system  for 
navigation  driven  from  voice  recognition,  much  improved  from  an  accuracy  of  35\%  correct  using 
the  standard  earlier  model.  Very  recently,  these  results  have  been  expanded  by  Magerman  and  others 
working  with  him  at  EBM  and  at  BBN,  achieving  the  parsing  of  Wall  Street  Journal  text  with  95\% 
of  sentences  receiving  two  or  fewer  parsing  errors. 

Most  remarkably,  these  parsers  operate  with  no  hand-constructed  grammar  at  ail;  they  automatically 
extracts  all  information  from  a  corpus  of  pre-parsed  material.  The  most  recent  work  is  all  trained  on 
Perm  Treebank,  developed  by  the  author.  This  treebank  now  consists  of  5.5  million  words  of  text 
hand  tagged  for  part-of-speech,  with  about  two-thirds  of  this  material  also  hand  annotated  with  a 
skeletal  syntactic  bracketing.  This  material  has  already  been  used  for  purposes  ranging  from  serving 
as  a  gold-standard  for  parser  testing  to  serving  as  a  basis  for  the  induction  of  stochastic  grammars  to 
serving  as  a  basis  for  quick  lexicon  induction. 

Lexical  Semantics  and  Beyond 

Initially,  it  would  appear  most  unlikely  that  statistical  techniques  would  be  of  much  use  for  either  the 
discovery  or  representation  of  the  meanings  of  words.  Surprisingly,  a  unsupervised  training  method 
has  recently  been  devised,  whose  performance  equals  supervised  methods,  promising  much  less 
dependence  on  large  annotated  corpora  of  materials  for  training  purposes.  This  opens  up  the 
possibility  that  ML  components  for  languages  other  than  English  will  become  easy  to  develop, 
requiring  primarily  large  collections  of  unnannotated  text. 

Penn  has  now  achieved  word-sense-disambiguation  results  of  96.5\%  correct  using  unsupervised 
training  methods.  Resolving  word  sense  ambiguity  requires  determining  the  proper  reading  of  a  given 
word  of  text.  This  problem  is  crucial  in  lexical  selection  in  language  translation,  in  determining  the 
correct  pronunciations  of  homophonous  words  within  a  text-to-speech  system,  and  in  restoring 
accents  in  7-bit  ASCII  encoded  texts  in  languages  such  as  French  or  Spanish.  A  new  essentially 
unsupervised  training  method  developed  by  Yarowsky  achieves  96.5\%  correct 
word-sense-disambiguation  on  a  reserved  test  corpus.  This  algorithm  requires  only  a  standard 
dictionary  definition  for  each  of  the  two  word  senses  as  a  seed. 

In  short,  the  shift  in  methodology  within  NLP  is  now  producing  robust  system  components  which 
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perform  very  weO  on  unconstrained  language  tasks.  These  components,  when  used  in  environments 
of  limited  vocabularies  and  task  complexities,  should  allow  extremely  low  error  rates,  particularly 
with  users  who  have  become  accustomed  to  system  limitations. 
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WORKING  GROUP  REPORTS 


BATTLE  COMMAND 
BATTLELAB 


In  addition  to  discussing  the  three  application 
areas  considered  by  all  of  the  working  groups 
(training,  assistance,  and  operational)  this 
group  also  dealt  with  issues  related  to 
research,  integration,  and  engineering. 


Spoken  Human-Machine  Dialog  Workshop  1995 
Battle  Command  Working  Group  Findings 
POC:  CPT  Kelly  Lauritzen,  Cliff  Weinstein 


Overview 

Our  working  group  focused  on  potential  areas  of  future  development  in  voice  technology  in  three 
areas;  training,  assistance  and  operational  applications. 


Training  Applications 


Conversation  is  so  wide  ranging  that  it  presents  difficult  in  voice  applications  just  be  the  sheer 
volume  of  possible  utterances.  Training  applications  have  an  advantage  in  that  the  application  can 
often  control  and  focus  the  conversation.  It  can  proceed  in  a  step  by  step  manner  in  training  the 
soldier.  Here  are  several  high  pay-ofif  areas: 


Foreign  language  training;  Foreign  language  specialists  require  training  that  is  time  and  personnel 
intensive.  Voice  technology  could  help  soldiers  to  practice  interrogation,  translation  and  working 
with  coalition  forces.  It  could  also  help  foreign  language  specialists,  who  are  in  low  density  areas 
to  maintain  their  language  proficiency. 

Classroom  XXI:  As  the  Army  transitions  to  Classroom  XXI,  voice  technology  has  a  possible  role 
in  multimedia  and  distance  learning.  However,  it  is  not  clear  how  well  matched  it  is  to  the 
environment.  Since  most  classrooms  are  not  simple  querys  and  answers,  but  more  complex 
interactions,  this  is  an  ill-defined  area.  It  does  appear  that  voice  output  will  be  important  factor  in 
Classroom  XXI 

Simulations:  setting  up  training  applications  is  a  data  intensive  operation.  It  requires  putting  in 
units,  routes,  activities,  etc.  This  could  possibly  be  done  better  with  voice.  It  also  presents  an 
interesting  problem  with  the  combination  of  voice  with  a  graphical  environment,  since  many 
military  training  applications  focus  on  maps,  map  features  and  unit  locations.  Finally,  in  the 
military,  we  try  to  train  as  we  fight,  so  these  training  systems  are  best  when  integrated  into  actual 
command  and  control  systems. 

Assistance  Applications 


We  viewed  information  help  services  as  the  area  with  the  most  potential  for  the  application  of  voice 
technology.  Often  when  you  need  assistance,  both  your  hands  and  eyes  are  occupied.  Fbdng  an 
engine  or  radio,  you  might  be  in  the  middle  of  a  procedure  and  suddenly  need  assistance  or  an 
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answer  to  a  question.  Being  able  to  ask  the  computer  right  then  would  save  considerable  effort. 
You  could  also  include  ordering  procedures,  so  when  you  noted  that  you  needed  a  part,  you  could 
ask  the  computer  to  help  to  identify  the  part  number  and  then  to  order  it.  Because  the  domain  is 
limited,  you  could  limit  the  scope  required  for  the  grammar  and  vocabulary.  Other  help  services 
could  be  offered  over  phone  lines  as  is  currently  done  on  a  limited  scale.  This  same  system  could 
include  voice  authentication  to  control  access. 

A  language  based  search  aid,  could  help  you  in  conversations  in  foreign  languages.  There  are 
currently  many  foreign  language  aids,  where  you  can  type  in  the  word  in  one  language  and  see  the 
translation.  This  same  application  would  be  much  more  efficient,  if  you  could  speak  the  word  into 
the  device  and  then  hear  what  the  translation  is.  These  would  be  especially  applicable  in  technical 
or  military  ^plications,  where  even  a  person  who  is  quite  proficient  in  a  foreign  language  might  not 
know  the  technical  terms.  This  is  intuitively  obvious  to  most  people  because  often  native  speakers 
have  trouble  listening  to  discussions  with  many  technical  terms.  This  would  not  only  include  single 
words,  but  idiomatic  and  common  phrases. 

Other  assistance  application  possibilities  are  applications  for  controlling  vehicles  or  devices.  A 
Global  Positioning  System  could  warn  you  verbally  if  you  strayed  too  far  from  a  given  route.  It 
might  also  be  used  to  control  vehicles  directly  like  an  Unmanned  Aerial  Vehicle. 


Operational  Applications 


Voice  is  the  logical  choice  for  Command  and  control  on  the  move.  When  moving  in  a  command  and 
control  vehicle  cross  country,  it  is  impossible  to  use  a  mouse,  keyboard  or  touch  screen  as  an  input 
device.  Here  noise  is  probably  a  major  factor,  but  we  noted  that  if  the  noise  frequency  range  from 
the  engines  was  outside  of  the  voice  range,  it  might  not  be  so  critical. 


In  the  same  way  voice  is  suited  for  training  in  foreign  languages,  it  also  has  potential  as  a  translation 
device  for  Joint  Forces.  This  would  ease  the  need  for  trained  language  specialists  and  could  provide 
a  real  time  translation 


Covert  Operations  often  require  overheading  conversations  and  communications.  This  would 
include  both  natural  language  processing  and  voice  recognition.  If  done  successfully,  it  would 
facilitate  continuous  monitoring  without  being  so  labor  intensive. 


Voice  Identify  Friend  or  Foe.  Currently,  using  challenge  and  password  is  a  risky  and  dangerous 
operatioa  If  you  could  improve  voice  matching  technology,  you  could  use  a  database  as  a  security 
measure  to  identify  persormel  before  admitting. 
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Engineering  and  Integration  Issues 


Voice  ^^plications  often  appear  to  be  less  successful  than  the  technology  actually  is  because  people 
expect  applications  to  be  “smarter”  with  voice  input.  To  get  a  truer  picture,  personnel  using  voice 
systems  need  to  have  familiar  training  in  both  the  voice  system  and  target  application  to  be 
successful.  This  reflects  the  lack  of  robustness  and  lack  of  integration  of  artificial  intelligence  in 
voice  technology.  In  other  words,  voice  is  viewed  not  only  as  an  input,  but  a  higher  level  of 
intelligence  in  the  application  itself 

A  ^}eedi  Hataha.<ie  for  the  military  environment  would  be  useful  in  testing.  For  example,  there  could 
bereccHxiing  of  radio  transmissions  to  test  recognition  technology  against  the  environment  it  would 
be  used  in.  Other  databases  could  be  vehicle  or  weapon  noise  signatures. 


Integrating  a  voice  recognition  system  into  another  application  requires  more  than  just  an  efficient 
voice  recognizer.  The  speech  technologist,  system  contractor  and  government  representative  must 
establish  a  close  working  relationship.  This  is  a  project  management  note  and  does  not  relate 
directly  to  voice  technology,  but  often  has  a  dramatic  effect  on  the  success  of  the  project. 


Vocabulary  and  grammar  must  be  easy  enough  to  change  and  modify  that  the  end-user  can  maintain 
or  change  them.  Ultimately,  a  toolkit  that  would  allow  non-technicians  to  develop  a  grammar  and 
vocabulary  would  be  the  best  solution. 


A  major  advantage  of  voice  access  is  that  it  leaves  the  eyes  and  hands  free  to  accomplish  other  tasks. 
This  raises  two  issues:  One,  voice  access  must  be  mobile.  If  tied  by  a  cable  to  a  computer,  users 
are  much  less  likely  to  use  voice  technology  because  it  would  require  donning  and  removing  a 
headset.  In  addition,  a  user  might  want  to  query  his  computer  while  looking  at  a  larger  map  display 
or  when  in  another  location.  Because  a  “no  tether”  solution  seems  most  desirable,  investigating 
possible  enemy  interference  wireless  and  infrared  effect  and  countermeasures  will  be  a  future 
engineering  issue.  Second,  in  order  to  keep  your  hands  free,  voice  technology  with  key  words. 


12 


Training  Applications 
o  Air  Traffic  Control  using  simulators 
o  Foreign  Language  Training 

-  Interrogation 

-  Translation/Transcription 

-  Coalition  Forces 

-  Maintenance 

o  Command  Operations  Training 

-  Part  of  Multimedia  lessons  in  Force  XXI  classroom 

-  Not  clear  how  well  matched 

-  Voice  output  important 

o  Control  and  setup  of  simulations  for  wargaming 

-  Leathemet 

-  JANUS,  CBS 

-  Mission  planning,  Rehearsal,  Training  System 
o  System  Maintenance  &  Repair 

o  Train  as  you  fight 

o  Training  on  voice  systems  which  are  to  be  integrated  into  C2 
Systems 
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Assistance  Applications 
0  Maintenance  of  Radios 

o  Maintenance  &  Debugging  of  information  systems 
o  Automated  Help  Desk 
o  Automated  Information  Service 

-  All  voice  -  Jane’s 

-  Authentication  including  voice  for  access  control 
o  Language-based  information  search  aids 

o  Advanced  Language  Assistants 

-  Technical 

-  Military 

o  Direction  Assistance 

-  Warning 

o  Remote  control 

-  UAV  (Unmanned  Aerial  Vehicle) 


Operational  Applications 


0  C20TM 
0  Phoenix 

0  Translation  for  Joint  Forces 
o  Direction  Finding 

-  GPS  ->  voice  output 

-  Voice  interrogation  exercises 

0  Covert/Adverse  Operations  &  Communications 
o  Voice  IFF 


R&D  Issues 

o  Voice  Recognition  on  wireless  LANs 

-  in  C2Vs,  for  example 
.  in  TOC 

o  Grammar  design  and  control 
o  Robust  performance  in  noise  and  stress 
o  Soft  or  whispered  speech 
o  Special  Operations  Forces  (SOF) 
o  Wordspotting  to  get  rid  of  PTT 
o  listing  to  make  queries  more  flexible 

-  Robust  grammars 

o  Database  creation  &  capture 
o  Toolkits 

-  Techie 

-  User 

o  Microphone  Robustness 
o  Out-of-Phraseology  Handling 

-  Out-of- Vocabulary 

-  User  setting  on  confirmation 

-  Rejection 
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Engineering  &  Integration 


o  Training  in  use  of  voice  systems 

o  Speech  database  collection  for  the  environment 

o  Integration  of  software  based  recognition  technology  in  C2 
systems 

-  Phoenix 

-  Speech  Technologist/System  Contractor/Govemment 
o  Change  and  control  of  vocabulary  &  grammar 

o  Flexibility  &  Tailorability 
o  Toolkits  (usable) 
o  Ruggedness  &  EMI  concerns 
o  Different  levels  of  human/machine  interface 

-  Engineer 

-  Administrator 

-  User 

o  Mobility  of  voice  access  -  no  tether 

-  wireless/infi-ared  interference  effect  on  recognition 
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MOUNTED  WARFIGHTING 
BATTLESPACE 


NUMBER:  1 


SPOKEN  HUMAN-MACHINE  DIALOGUE  WORKSHOP 

PANEL:  MOUNTED  WARFIGHTING  BATTLESPACE. 

EVALUATION  AREA:  ASSISTANCE. 

TITLE:  CONFIRMATION  OF  MESSAGE  RECEIVED 

ANALYST:  CHUCK  CAMPBELL  <&  MARTIN  BOSEMER 

MBBL,  FORT  KNOX,  KY  40121  DSN:  464-1963/2045 

WARFIGHTING  MERIT  (VALUE  ADDED  TO  SOLDIER) 

The  soldier  is  accustomed  to  initiating  and  responding  to  verbal  communications  more  so  than  any 
other  form  of  communication.  The  ability  of  a  computer  system  to  respond  to  voice  communications 
is  extremely  important  and  critical  to  future  force  success  and  survivability  in  combat.  This 
technology  will  also  greatly  enhance  the  effectiveness  and  efficiency  of  operations,  training  and 
administration.  However,  just  as  with  humans,  the  soldier  must  ensure  that  the  computer  received 
the  information  and  is  going  to  react  just  as  the  sender  intended.  A  simple  "ROGER"  as  an 
immediate  answer  will  not,  in  most  cases,  be  an  acceptable  response  to  a  verbal  message  given  since 
there  is  no  assurance  that  the  computer  made  a  proper  interpretation  of  the  message. 

Effective,  concise,  and  timely  communications  are  critical  to  the  order  of  battle  and  in  every  day 
actions.  Commanders/leaders  at  all  levels  must  be  fully  aware  of  the  intent  of  a  message  received  and 
must  convey  to  the  sender  that  the  message  was  received,  understood,  and  that  the  desired  response 
will  occur.  Computer  systems  inherently  have  the  ability  to  verify  accurate  receipt  of  data 
transmissions,  detect  and  correct  transmission  errors  with  less  than  a  0.0000001  to  1  chance  of  error. 
However,  computers  today  are  only  maintaining  a  84%  success  rate  for  accurately  interpreting 
fairly  simple  voice  communications.  With  the  capability  of  spoken  human-machine  dialogue, 
commanders,  leaders,  staffs  will  become  party  to  increased  volumes  of  simultaneous  verbal 
communications  which  will  become  ever  increasing  emotionally  sterile  verbiage.  Ensuring  that  the 
proper  commands/actions/reactions  are  obtained  and  that  the  emotional  characteristics  of  human 
communications  are  compensated  for,  will  require  an  initial  response  more  descriptive  than 
"ROGER". 

The  desired  voice  response  fi'om  a  computer  should  contain  information  which  ensures  the  sender  that 
the  receiver  correctly  heard  the  information  and  is  going  to  provide  the  reaction  desired  by  the  sender. 
The  communications  from  the  computer  must  address  which  vocal  information  it  is  responding  to 
and,  in  action  summary  form,  inform  sender  of  action  to  be  taken.  System  must  react  to  a 
"NEGATIVE"  type  statement  to  halt  a  reaction  which  was  misinterpreted  or  to  carry  on  with  the 
acceptance  of  a  default  response  statement  similar  to  "ROGER "  or  "OK". 

Spoken  communications  with  a  computer  or  group  of  computers  is  essential  in  maintaining  pace  with 
the  ever  increasing  speed  and  volume  of  information.  Although  the  average  processing  time  of 
information  has  been  drastically  reduced  over  the  past  20  years,  the  volume  of  information  processed 
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has  increased  at  a  greater  relative  rate.  Spoken  human-machine  dialogue  is  the  next  step  required  to 
maintain  the  pace  until  direct  interface  with  the  human  mined  is  attained. 

Initial  and  continued  importance  must  be  placed  on  system  development  which  will  improve  the 
capability  of  the  computer  to  separate  simultaneous  voice  traffic,  identify  which  voice  traffic  it  should 
respond  to,  provide  short  and  logical  responses  to  the  sender  ensuring  the  pending  action  is  that 
desired  by  the  sender,  and  waiting  a  short  pre-determined  period  of  time  for  a  correction  statement 
before  accepting  a  default  of  "OK".  Further  value  would  be  in  the  computers  ability  to  analyze  the 
voice  pattern  to  ascertain  the  immediacy  of  the  action  as  indicated  by  the  volume  and  intensity  of  the 
voice  or  expletive  used  in  the  communication.  The  reduced  time  required  for  information  entry  by 
humans  will  certainly  accelerate  the  volume  of  information  processed.  The  greatest  challenge  to  be 
faced  with  spoken  human  machine  interface  will  be  the  dramatic  increase  in  information  processing 
and  subsequent  upgrades  in  computer  systems  for  processing  requirements  since  humans  tend  to  over 
use  systems  which  are  easy  to  use.  Will  the  time  required  for  a  computer  or  person  to  provide  a 
logical  response  be  acceptable?  Probably  not.  In  time  the  traditional  "ACKNOWLEDGED", 
ROGER"  or  "OK"  will  be  commonly  used  as  a  response  for  most  applications. 

Application  of  spoken  human  machine  dialogue  is  not  a  question  of  "if  it  becomes  available",  it's  a 
question  of  "when  will  I  be  able  to  apply  the  technology  to  my  needs?"  Incorporation  into  training, 
operations,  and  administration  can  be  accomplished  easier  and  at  a  faster  pace  than  incorporation  into 
combat  environments.  Simply  stated,  the  stable  physical  environment  and  repetitive  nature  of 
operations,  training,  and  administrative  tasks  are  normally  easier  and  more  cost  effective  to  be 
adapted  to  this  technology.  Additionally,  automation  users  are  not  balking  at  incorporating  this 
technology,  in  fact  there  is  high  demand/desire  for  spoken  interface  to  eliminate/minimize 
keyboard/video  interface.  Advances  in  spoken  human  machine  dialogue  are  commonly  occurring  in 
banking,  telephone  systems,  and  other  areas  with  great  acceptance  over  the  touch-tone  key  pad  entry. 

In  summary,  a  descriptive  confirmation  of  message  received  should  be  required  at  least  until 
technology  in  artificial  intelligence  and  voice  pattern  detection/separation  improve  to  point  where 
the  confidence  level  in  the  computers  ability  to  reason  approximates  or  exceeds  that  expected  of  a 
highly  trained  and  capable  soldier. 
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Spoken  Human-Machine  Dialogue  Workshop 


Title:  COM  message  extraction,  gisting,  and  screening 


Concept:  Develop  the  ability  to  screen  messages  for  relevant  communication,  retract  the 
desired  and  relevant  information,  and  then  gisting  (summarizing)  it  and  them  transmitting 
it  audibly  or  digitally. 

Technical  Assessment:  (Desired  Capabilities,  Current  Shortfalls) 

On  the  simplest  level  it  would  consist  of  searching  the  COM  for  word  strings,  and  then 
play  audio  with  relevant  facts.  Advanced  would  include  context^tent  level  search  at 
COM  and  then  based  current  mission  context.  Summarize  it  to  the  desire  present  if  when 
the  pilot  needs  h. 

Warflghting  Merit:  (Value  added  to  Soldier) 

It  should  substantially  reduce  workload  of  crewmen  in  tanks  and  helicopters  and  reduce 
the  overaU  “chuttles”  in  the  cockpit.  By  automating  the  scanning  a  computer  could  scan 
multiple  channels  for  relevant  data. 

Current  Projects:  Unknown. 

Required  Research:  Advanced  development  at  implementing  word  seats  in  a  COM 
environment  (short  term)  (long  term)  would  require  complete  development  at  a  natural 
language  cap^ility  (to  include  assessment  of  content  and  intent  in  real  time).  Since  this 
technology  would  already  include  an  understanding  of  contents  and  intent  to  do 
assessment,  it  should  be  easy  to  summarize  information  for  crewmen  or  pilot. 


Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Mounted  Warfiehting  Battlespace 


Assistance 
iTLie  Quc^ 


Operations 


Title:  Confirmation  of  message  received 

Concept:  When  a  human  talks  into  a  voice  recognition  system,  the  system  must  confirm 
that  is  heard  what  was  said.  Sometimes  “Roger”  is  enough.  Sometimes  confirmation  is 
needed  that  the  message  heard  was  what  the  human  spoke.  If  a  human  speaks  and  many 
seconds  pass  without  acknowledgment  by  the  computer,  the  human  tends  to  become 
frustrated. 


Technical  Assessment:  (Desired  Capabilities,  Current  Shortfalls) 

Very  easy  to  do.  There  are  some  interesting  questions  as  to  when  to  provide  feedback  by 
voice  synthesis,  when  by  screen  display,  when  by  other  modalities;  and,  how  the  voice 
recognition  system  would  decide  whether  to  say  “Roger”  or  to  speak  back  exactly  what  it 
heard  or  to  speak  back  a  paraphrase.  It  seems  likely  that  answering  these  questions  on  a 
case-by-case  common  sense  basis  will  be  adequate. 

Warflghting  Merit:  (Value  added  to  Soldier) 

Increased  human  acceptability  of  voice  recognition  systems. 

Current  Projects:  N/A.  This  is  easy  to  do.  Many  systems  do  it.  All  need  to. 

Required  Research:  None  seems  needed. 
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NUMBER:  2 


SPOKEN  HUMAN-MACHINE  DIALOGUE  WORKSHOP 

PANEL:  MOUNTED  WARFIGHTING  BATTLESPACE. 

EVALUATION  AREA:  ASSISTANCE. 

TITLE:  CAUTIONS,  WARNINGS,  AND  ADVISORIES  (CWA) 

ANALYST:  CHUCK  CAMPBELL  &  MARTIN  BOSEMER 

MBBL,  FORT  KNOX,  KY  40121  DSN:  464-1963/2045 

WARFIGHTING  MERIT  (VALUE  ADDED  TO  SOLDIER) 

Computer  generated  cautions,  warnings,  and  advisories  are  a  vital  aspect  of  technology  today  and  in 
the  future.  Above  all  they  must  be  respected  as  an  integral  part  of  the  crew/platform  interface  and 
not  perceived  as  a  nuisance.  CWAs  are  ever  present  operational,  administrative  and  especially  in 
training  environments.  However,  interactive  voice  CWAs,  which  can  answer  analytical  questions 
you  pose  after  the  initial  CWA  are  required  with  emerging  technologies.  If  you  receive  a  low  fuel 
warning,  it  would  be  logical  to  ask  and  receive  response  to  how  much  is  left,  estimated  distance 
supported,  is  consumption  above  normal  level  possibly  indicating  a  leak,  etc.  CWAs  must  become 
interactive  and  analytical  as  they  are  included  in  future  technologies  to  ensure  success  of  future  force 
projection  and  survivability  during  combat. 

As  most  of  us  are  aware,  CWAs  are  ever  present  in  our  every  day  life  from  smoke  detectors,  various 
car  warnings,  and  especially  in  the  fast  food  industry.  The  common  tone  CWA  is  rapidly  being 
replaced  with  voice  CWAs.  The  car  industry  was  one  of  the  first  industries  to  begin  replacing 
non-critical  tone  CWAs  with  voice  CWAs.  Almost  everyone  respects  a  person  who  has  little  to  say, 
but  is  eloquent,  effective,  and  convincing  when  they  do  speak.  To  be  successful,  CWAs  must  also 
posses  this  quality  as  opposed  to  being  perceived  as  an  annoyance. 

The  system  must  be  able  to  discern  the  criticality  of  the  event  and  to  adjust  the  voice  output 
commensurate  to  the  need  for  attention  from  the  intended  receiver.  This  "emotional"  characteristic 
may  not  appear  critical,  however,  one  would  anticipate  that  personnel  subjected  to  long  periods  of 
monotone  communications  with  a  computer  would  loose  valuable  seconds  in  reacting  at  critical  times. 
Efforts  within  aviation  have  proven  that  audible  alerts  must  vary  with  the  criticality  of  the  situation 
to  stimulate  the  crew  into  an  enhanced  reactive  state.  The  merits  of  CWAs  cannot  be  minimized. 
The  relevant  factor  to  the  success  of  such  systems  is  to  determine  on  a  varying  scale,  when  CWA 
is  needed  and  what  intensity  level  of  audio/visual  CWA  is  needed  to  communicate  the  level  of 
urgency. 

System  must  also  be  sensitive  enough  to  minimize/eliminate  normal  information  which  may  become 
insignificant  due  to  battle  conditions  or  platform  component  loss.  Crews/users  are  not  concerned  that 
a  scheduled  service  is  required  due  to  miles  or  days  when  engaged  in  combat.  The  system  should 
hold  this  type  of  information  until  a  more  reasonable  time  or  information  requested.  This  could  be 
accomplished  simply  with  a  scaled  over  ride  based  on  crews/users  assessment  of  the  mission  mode 
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and  tempo.  Finally  the  system  cannot  overburden  the  crew/user  with  constant  information  or  the 
crew/user  will  mentally  tune  the  system  out  if  not  physically  turning  the  volume  down  or 
disconnecting  the  device,  reactions  typical  to  today's  automobile  CWAs.  If  you  dislike  the  tone  or 
voice,  especially  reminders  to  fasten  your  seat  belt,  disconnect  it. 

CWAs  can  be  separated  into  three  general  categories  of  platform  performance/environment  (to 
include  resource  availability),  external  environmental  conditions,  and  external  threats.  Within  these 
three  categories,  CWAs  can  be  classified  as  advisory  or  routine  update,  caution,  warning,  and 
imminent.  CWAs  are  typical  in  today’s  technology  but  fijture  technologies  must  posses  the  capability 
for  interactive  spoken  human-machine  interface.  Adding  voice  interface  will  minimize  visual  sensory 
interruptions,  such  as  an  operator  having  to  visually  see  gauges  or  setting/selections  to  identify  a 
potential  problems.  However,  a  balance  between  visual  and  audio  CWAs  is  critically  important, 
especially  during  combat.  While  verbal  CWA  allows  increased  visual  capabilities  to  monitor  the 
threat,  combat  also  brings  additional  voice  communications  both  internal  to  the  platform  and  external 
with  which  the  CWA  must  compete  for  crew/user  attention  or  may  even  become  an  unwanted 
interruption. 

Although  spoken  human-machine  dialogue  is  just  emerging  as  an  effective  method  of 
waming/problem  identification  and  is  only  on  the  frontier  of  being  dependable  in  the  subsequent 
solution/resolution  process,  development  and  use  will  be  essential  in  all  future  technology  involving 
automation. 
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NUMBER:  3 


SPOKEN  HUMAN-MACHINE  DIALOGUE  WORKSHOP 

PANEL:  MOUNTED  WARFIGHTING  BATTLESPACE. 

EVALUATION  AREA:  ASSISTANCE. 

TITLE:  COMMUNICATION  MESSAGE  EXTRACTION, 

GISTING,  AND  SCREENING. 

ANALYST:  CHUCK  CAMPBELL  &  MARTIN  BOSEMER 

MBBL,  FORT  KNOX,  KY  40121  DSN:  464-1963/2045 

WARFIGHTENG  MERIT  (VALUE  ADDED  TO  SOLDIER) 

The  volume  of  communications,  information,  and  data  is  increasing  at  an  exponential  rate.  The  U.S. 
has  witnessed  an  unprecedented  growth  in  data  processing  over  the  past  twenty  years.  Managers  no 
longer  have  the  ability  to  review  transaction  ledgers,  only  summaries.  We  have  evolved  into  a 
"BOTTOM  LINE  MANAGEMENT"  age  due  to  the  ever  increasing  complexity  and  volume  of 
information.  With  ever  increasing  volumes  and  types  if  information/data  sources,  computer  systems 
commonly  assimilate  information  into  short  answers.  Communication  message  extraction,  gisting, 
and  screening  is  predominantly  accomplished  by  administrative,  operational,  and  staff  personnel  on 
a  daily  basis.  In  almost  every  environment,  topics  and  bullet/paragraph  supporting  information  is  used 
when  higher  levels  of  management  are  being  informed.  Effective  and  concise  communication  message 
extraction,  gisting,  and  screening  will  be  required  from  spoken  human  machine  dialogue  and  advanced 
artificial  intelligence  systems  in  future  warfighting  situations  and  mounted  platform  operations  for 
successful  combat/operational  missions. 

Automation  is  another  area  where  summaries,  gisting,  screening,  and  extraction's  are  common.  In 
most  cases  files/messages  are  not  only  given  a  name  but  also  a  subject,  and  summary  of  contents  for 
potential  viewers  to  scan  for  interest.  Predominant  on  computer  networks/bulletin  boards  is  the 
ability  to  screen,  scan,  gist,  to  find  information  on  a  desired  topic. 

The  system  must  be  able  to  discern  the  criticality  of  the  event  and  to  adjust  the  level  of  detail  provided 
commensurate  to  the  need  by  the  intended  receiver.  This  "LOGICAL"  characteristic  may  not  appear 
critical,  however,  one  would  anticipate  that  personnel  subjected  to  large  quantities  of 
information/data,  over  long  periods  of  time,  would  loose  the  ability  to  maintain  an  overview  of  the 
situation.  Efforts  within  aviation  have  proven  that  detailed  platform  system  information  must  be 
available  to  the  crew  upon  request,  but  the  crews  rely  on  the  systems  to  provide  summary  information 
and  critical  warnings,  alerts,  and  advisories  based  on  previously  defined  parameters.  With  platform 
crew  sizes  diminishing,  information/data  increasing,  future  communication  systems  must  possess  the 
ability  to  screen,  gist,  summarize  information/data  if  the  platform,  commander,  trainer,  mechanic,  etc. 
are  to  be  effective.  They  must  be  able  to  have  variable  control  of  the  system  sensitivity  to  attain  the 
"appropriate  level  of  information  for  the  varying  mission,  environmental,  physical,  and  combat 
environment". 
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The  merits  of  communications  gisting,  extraction  and  screening  cannot  be  minimized.  The  relevant 
factor  to  the  success  of  such  systems  is  to  determine  on  a  varying  scale,  when  required  and  what 
intensity  level  of  summary,  etc.  is  needed  to  communicate  the  level  of  urgency.  Systems  must  posses 
selective  screening  algorithms  with  user  defined  parameters,  sensitive  to  both  the  perceived  criticality 
of  the  sender  and  the  receiver.  Criticality  level  cannot  be  a  simple  numeric  value,  on  the  traditional 
1  to  10  scale,  which  is  established  by  the  sender.  A  large  percentage  of  the  senders  will  be  parochial 
and  really  believe  their  commo  is  a  10  without  considering  the  complexity  of  the  overall  situation. 
On  the  other  hand,  the  receiver  may  consider  one  small  piece  of  information,  which  the  sender 
considered  a  1  to  3,  the  missing  link  in  the  picture.  Every  patient  and  sales  person  will  tell  the 
secretary/receptionist/nurse  at  the  doctors  office  that  they  are  the  most  important  person  the  Doctor 
needs  to  see.  What  makes  one  think  it  would  be  different  with  computers.  It  appears  that  human 
screening,  gisting,  extraction  will  be  required,  assisted  with  computerized  parameter  theory  systems, 
to  process  the  preponderance  of  information/data  until  artificial  intelligence  meets  requires  standards 
for  applicable  missions. 


Artificial  intelligence,  with  variable  parameters  established  by  the  user  or  system  managers,  is  required 
to  keep  pace  with  today's  information  flow.  The  absence  of  artificial  intelligence  will  be  a  critical 
limiting  factor  to  future  technological  advances  which  minimize  the  size  of  human  crews/staffs  while 
increasing  the  amount  and  types  of  critical  information  to  be  received,  processed,  interpreted,  reacted 
to,  trained,  or  forwarded. 
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NUMBER:  4 


SPOKEN  HUMAN-MACHINE  DIALOGUE  WORKSHOP 

PANEL:  MOUNTED  WARFIGHTING  BATTLESPACE. 

EVALUATION  AREA:  ASSISTANCE. 

TITLE:  ACCOMPLISHMENT  OF  ADMINISTRATIVE  TASKS 

ANALYST:  CHUCK  CAMPBELL  &  MARTIN  BO SEMER 

MBBL,  FORT  KNOX,  KY  40121  DSN:  464-1963/2045 

WARFIGHTING  MERIT  (VALUE  ADDED  TO  SOLDIER) 

Spoken  human  machine  dialogue  is  a  natural  progression  from  keyboard  entry  systems.  It  will  be  a 
natural  interaction  to  obtain  information  required  for  the  preponderance  of  administrative  tasks  and 
to  produce  hard  copy  output.  In  future  technology,  written  correspondence  with  signatures  for 
authentication  will  be  replaced  by  spoken  human  machine  dialogue  using  a  voice  print  for  a  higher 
level  of  verification/authentication.  This  technology  will  revolutionize  the  administrative  processes 
of  today,  if  not  achieved,  current  administrative  methods  will  rapidly  become  too  slow  for  processing 
the  flow  if  information/data  which  will  have  a  devastating  impact  in  combat.  History  has  documented 
and  criticized  many  instances  where  poor  administrative  capabilities  were  the  predominant  factor  in 
mission  failures.  Example  of  such  a  critical  administrative  delay  was  that  in  which  the  Japanese 
Ambassador  to  the  U.S.  was  late  in  delivering  the  Japanese  declaration  of  war  on  the  U.S.  prior  to 
the  bombing  of  Pearl  Harbor. 

Sever  backlogs  in  information  input  systems  was  a  significant  problem  to  the  banking  industry  in  the 
late  70s.  Huge  backlogs  in  information  input  and  verification  systems  frequently  caused  over  four 
days  delay  in  processing  check/credit  card  transactions  if  local  and  over  five  if  outside  the  region. 
Now  electronic  point  of  sale  interface  with  the  financial  clearing  systems  identify  funds  for  transfer 
immediately  at  the  time  of  sale.  Without  this  improvement  in  the  system.  Soon  your  automatic  teller 
may  not  require  a  card,  just  your  voice  and  security  information. 

This  technology  will  revolutionize  administrative  systems.  As  we  move  further  toward  a  paperless 
environment  we  must  also  move  towards  a  keyless  environment.  Spoken  human  machine  dialogue 
is  just  the  next  logical  step  required  towards  direct  mental  interface  between  humans  and  machines. 
Our  views  of  current  administrative  duties  must  be  replaced  with  the  vision  of  a  small  wireless 
headset/ear  device  and  microphone,  which  are  capable  of  independently  determining  which  voice 
data  is  intended  for  transmission,  destination(s)  desired,  additional  data  to  be  attached  to 
transmission,  most  effective  and  economical  method  of  transmission,  security/encryption  levels 
required,  interpretation  of  information  originating  in  a  language  other  than  the  users,  and  automatic 
switching/multi-channel  simultaneous  routing  of  information. 

The  financial,  legal,  and  proof  positive  requirements  have  generated  an  immense  requirement  for 
every  profession  to  produce  and  maintain  large  volumes  of  "hard  copy  documentation".  Instilling 
a  sociological  reform  towards  "hard  copy  documentation"  will  be  the  greatest  challenge  in  the 
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emergence  of  this  technology  within  administrative  areas.  As  an  example,  the  technology  of  manually 
typed  transcription/recording  services,  commonly  used  in  court  rooms,  are  being  parallel  with  court 
cameras  and  recording  devices,  but  not  replaced.  Courts  still  require  court  recorders.  Until  voice 
recognition  systems,  high  volume  data  storage,  enhanced  information  retrieval  systems,  and  of  most 
importance  relational  network  interface  systems  for  information  sharing/routing  are  perfected  and 
accepted  as  an  "OFFICIAL  RECORD",  benefits  will  be  comparatively  minimum. 

The  financial,  legal,  and  proof  positive  requirements  have  generated  an  immense  requirement  for 
every  profession  to  produce  and  maintain  large  volumes  of  "hard  copy  documentation".  Instilling 
a  sociological  reform  towards  "hard  copy  documentation"  will  be  the  greatest  challenge  in  the 
emergence  of  this  technology  within  administrative  areas.  As  an  example,  the  technology  of  manually 
typed  transcription/recording  services,  commonly  used  in  court  rooms,  are  being  parallel  with  court 
cameras  and  recording  devices,  but  not  replaced.  Courts  still  require  court  recorders.  Until  voice 
recognition  systems,  high  volume  data  storage,  enhanced  information  retrieval  systems,  and  of  most 
importance  relational  network  interface  systems  for  information  sharing/routing  are  perfected  and 
accepted  as  an  "OFFICIAL  RECORD",  benefits  will  be  comparatively  minimum. 

Once  the  spoken  human  machine  dialogue  administrative  methods  are  perfected  and  accepted,  the 
frustration  of  "paperwork"  associated  with  job  performance  will  become  minimal.  In  addition, 
information  will  be  less  subject  to  error  based  on  multiple  transcriptions.  Future  technology  will  not 
be  able  to  emerge  without  the  incorporation  of  spoken  human  machine  dialogue  in  the  administrative 
areas. 
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Spoken  Human-Machine  Dialogue  Workshop 


Title:  Accomplishment  of  Administrative  Tasks 


Concept:  Input  and  query  of  personnel  data,  maintenance,  etc.,  by  voice  recognition.. 

Technical  Assessment:  (Desired  Capabilities,  Current  Shortfells) 

Desired:  Update  and  query  by  voice  rather  than  key-in. 

Warflghting  Merit:  (Value  added  to  Soldier) 

Faster  input  of  information,  so  fewer  soldiers  needed  for  administrative  task.  No  need  for 
soldiers  to  learn  to  be  good  typists  in  order  to  perform  admirustrative  tasks. 

Current  Projects:  COTS  tools  are  now  available  for  controlling  forms-oriented  inputs 
that  have  Mghly  restricted  syntax. 

Required  Research:  None  seems  needed. 


NUMBER:  5 


SPOKEN  HUMAN-MACHINE  DIALOGUE  WORKSHOP 

PANEL:  MOUNTED  WARFIGHTING  BATTLESPACE. 

EVALUATION  AREA:  ASSISTANCE. 

TITLE:  ADVANCES  IN  VOICE  SYNTHESIS 

ANALYST:  CHUCK  CAMPBELL  &  MARTIN  BOSEMER 

MBBL,  FORT  KNOX,  KY  40121  DSN:  464-1963/2045 

WARFIGHTING  MERIT  (VALUE  ADDED  TO  SOLDIER) 

Voice  synthesis,  in  the  development  of  spoken  human  machine  dialogue,  is  a  critical  factor  to  the 
overall  success  of  future  systems.  The  immense  complexity  of  developing  a  system  that  not  only 
recognizes  voice  generated  sounds  from  an  environment  with  a  multitude  of  various  sounds,  but  also 
detects  which  language  and  dialect  is  being  spoken  is  recognized  as  ambitious  but  necessary.  The 
translation  process  would  appear  simple  once  the  language  and  dialect  identification  process  is 
completed.  Generating  a  response  from  the  computer  would  be  considered  easier  since  the  language 
and  dialect  of  the  receiver  would  probably  be  known  or  given  as  a  known  variable. 

The  true  warfighting  merit  of  voice  synthesis  is  in  how  the  machine  developed  voice  conveys  the 
seriousness  of  the  information  being  presented.  The  voice  synthesis  process  cannot  be  monotone  and 
at  the  same  level  or  the  receiver  will  become  less  responsive.  There  must  be  an  "emotional"  quality 
to  the  output  to  be  considered  an  effective  crew  member.  With  the  capability  of  spoken 
human-machine  dialogue,  commanders,  leaders,  staffs  will  become  party  to  increased  volumes  of 
simultaneous  verbal  communications  which  will  be  emotionally  sterile  verbiage.  Ensuring  that  the 
proper  commands/actions/reactions  are  obtained  and  that  the  emotional  characteristics  of  human 
communications  are  compensated  for,  voice  synthesis  will  initially  have  to  be  more  descriptive  than 
normal  human  communications.  The  desired  quality  of  voice  synthesis  could  be  attained  if  the 
computer  had  the  ability  to  analyze  the  voice  pattern  to  ascertain  the  immediacy  of  the  action  as 
indicated  by  the  volume  and  intensity  of  the  voice  or  expletive  used  in  the  communication. 

In  summary,  humans  will  have  to  verbalize  the  criticality  of  the  information  being  presented  at  least 
until  technology  in  artificial  intelligence  and  voice  pattern  detection/separation  improve  to  point 
where  the  confidence  level  in  the  computers  ability  to  reason  and  express  emotional  characteristics 
approximates  or  exceeds  that  expected  of  a  highly  trained  and  capable  soldier. 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Mounted  Warfighting  Battlesoace 


Title:  Advances  in  voice  synthesis 


Concept:  Develop  fieldabie  technology  that  provides  man  natural  and  intelligible 
synthesis  of  speech  by  computer. 

Technical  Assessment:  (Desired  Capabilities,  Current  Shortfalls) 

—  Take  into  account  discourse  context  in  shaping  information  patterns. 

—  More  accurately  represent  idiosyncrasies  of  English  pronunciations. 

Warflghting  Merit:  (Value  added  to  Soldier) 

—  Adds  to  traiiwg  application  (SAPOR). 

—  Makes  computer  output  more  intelligible  in  noisy  environment. 

Current  Projects: 

—  COTS  available  but  doesn’t  sound  natural. 

—  e.g.  Barbara  Gross  experiments  in  using  discourse  knowledge  in  speech  synthesis 

—  Armstrong  Labs  ongoing  intelligibility  studies 

Required  Research: 

—  Discourse  context  representations  and  influence  in  prosody. 

—  Bigger  databases  of  word  pronunciation  in  English. 

—  Something  to  make  audio  output  more  human-like. 

These  cut  across  concepts:  Re-use 

Accessibility  (Modifiable  at  local  (end) 

Ihiman  Factoring 
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NUMBER:  6 


SPOKEN  HUMAN-MACHINE  DIALOGUE  WORKSHOP 

PANEL:  MOUNTED  WARFIGHTING  BATTLESPACE. 

EVALUATION  AREA:  ASSISTANCE. 

TITLE:  REFERENCE  ACCESS 

ANALYST:  CHUCK  CAMPBELL  &  MARTIN  BOSEMER 

MBBL,  FORT  KNOX,  KY  40121  DSN:  464-1963/2045 

WARFIGHTING  MERIT  (VALUE  ADDED  TO  SOLDIER) 

Reference  access  is  essential  to  developing  outstanding  training  and  operations  programs.  The  ability 
to  attain  related  information  on  a  subject,  in  a  verbal  discussion,  allows  for  a  more  consistent 
interactive  question  and  answer  session ,  minimizing  the  traditional  interruptions  and  countless  trips 
to  various  sources  of  information.  Of  greater  importance  operationally,  is  the  ability  of  the  soldier 
to  obtam  reference  material  in  the  maintenance  world  without  having  to  move  from  his/her  position. 
The  increased  ability  for  the  super  technicians  maintaining  this  advanced  equipment  will  only  be 
limited  by  their  ability  to  access  technical  information.  The  amount  of  information  required  for 
technical  maintenance  will  far  exceed  the  ability  of  the  human  mind  to  retain.  In  addition,  with  a 
smaller  army,  the  technicians  have  been  absorbing  more  and  more  systems  to  maintain  within  their 
MOS.  Spoken  human  machine  dialogue  will  achieve  it's  greatest  impact  within  the  interaction  of 
reference  information  and  assisting  in  determining  a  known  outcome  before  the  soldier  proceeds  with 
intuitive  maintenance  or  operational  operations  ("what  happens  if  I  do  this?"). 

The  ability  to  quickly  reference  information  systems  will  be  of  immeasurable  assistance  to  platform 
crew  members  as  they  have  to  perform  field  expedient  repairs.  The  computer  can  in  fact  lead  them 
step  by  Step  through  the  problem  analysis  and  identification  phase  on  through  the  actual  repair  phase. 
This  ability,  if  information  availability  exists  will  enhance  the  operational  capability,  survivability,  and 
sustain  ability  of  any  mounted  platform. 

The  uses  are  limitless  in  the  training  program,  especially  with  remedial  and  individual  paced  programs. 
This  one-on-one  environment/interface  can  adapt  to  the  logic  and  thought  patterns  of  the  student 
, without  the  fimstration,  based  on  the  questions  the  student  poses  to  the  system.  To  the  system  there 
are  truly  no  dumb  questions. 

The  ability  to  ask  a  question,  and  to  have  a  computer  research  various  sources  for  the  information, 
and  to  be  ^le  to  discuss  the  findings,  or  have  any  analytical  functions  performed  with  the  information 
available,  in  a  hands  free  mode,  will  have  immeasurable  positive  impacts  on  the  combat,  operational 
and  training  capabilities  of  the  Army. 
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spoken  Human-Machine  Dialogue  Workshop 


Panel:  Mounted  Warfighting  Battlespace 

Training  x'^sistance  Operations 

(  (Circle  One)  ) 


Title:  Reference  Access  (Spoken  Natural  Language  Reference  Access  to  Manuals,  etc. 
Orders) 

Concept: 

—  Easier  crew  access  to  compUcated,  obtuse  manuals 

—  Step-through  diagnosis  and  repair  from  manuals 

Technical  Assessment:  (Desired  Capabilities,  Current  Shortfalls) 

—  Must  have  manuals  digitized  and  in  computer-accessible  form. 

—  Fmrly  broad  vocabulary  natural  language  processing 

Warfighting  Merit:  (Value  added  to  Soldier) 

—  More  knowledge  available  to  sol(Uer  as  ne^ed 

—  Decreased  reliance  on  specialists 

—  Allows  hands-busy  eyes  busy  access  to  knowledge 

—  Save  space 

—  Permits  easier  updates 

—  Multimedia  (voice/sound/video/picture) 


Current  Projects: 

—  Some  work  being  done  in  di^tiring  numuals.  Air  Force  flight  control  systems 

—  Dialogue,  NLP,  voice  research 


Required  Research: 

—  Designing  format  for  storing  manual  knowledge 

—  Design  semantics,  grammar  for  accessing  this  knowledge  using  NLP. 

—  Make  updating  easy/automated. 
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NUMBER:  7 


SPOKEN  HUMAN-MACHINE  DIALOGUE  WORKSHOP 

PANEL:  MOUNTED  WARFIGHTING  BATTLESPACE. 

EVALUATION  AREA:  ASSISTANCE. 

TITLE:  VIRTUAL  CREW  MEMBER 

ANALYST:  CHUCK  CAMPBELL  &  MARTIN  BOSEMER 

MBBL,  FORT  KNOX,  KY  40121  DSN:  464-1963/2045 

WARFIGHTING  MERIT  (VALUE  ADDED  TO  SOLDIER) 

The  virtual  crew  member  is  an  absolute  requirement  which  we  are  slowly  evolving  towards  with 
today's  technology.  Technology  has  provided  for  many  sophisticated  independent  systems  which  are 
evolving  from  fixed  program  based  systems  to  developed  logic  based  systems  which  approach 
artificial  intelligence.  The  virtual  crew  member  is  one  that  appropriately  responds  with  the  correct 
action/reaction  within  less  time,  at  greater  efficiency,  and  at  a  level  exceeding  a  well  trained  and 
seasoned  veteran  soldier  while  interfacing  well  with  other  crew  members.  The  virtual  crew  member 
should  also  be  a  single  source  of  information  integration  from  independent  platform  systems.  Every 
system  should  operate  independently  so  damage  to  one  system  will  have  a  minimum  impact  on  other 
systems.  The  virtual  crew  member  should  not  only  perform  certain  functions,  such  as  the  loaders 
position  on  a  tank  (intelligent  autoloader),  but  should  also  be  able  to  integrate  information  exchange 
between  all  platform  systems. 

The  ability  to  quickly  evaluate  the  functional  status  of  platform  systems  and  to  quickly  attain 
reference  material  will  be  of  immeasurable  assistance  to  platform  crew  members  as  they  have  to 
perform  field  expedient  repairs.  The  computer  can  in  fact  lead  them  step  by  step  through  the  problem 
analysis  and  identification  phase  on  through  the  actual  repair  phase.  This  ability,  based  on 
independent  systems  information  connectivity,  will  enhance  the  operational  capability,  survivability, 
and  sustainability  of  any  mounted  platform. 

It  is  essential  that  the  virtual  crew  member  have  spoken  human  machine  dialogue  capability.  The 
soldier  is  accustomed  to  initiating  and  responding  to  verbal  communications  more  so  than  any  other 
form  of  communication.  The  ability  of  a  computer  system  to  respond  to  voice  communications  is 
extremely  important  and  critical  to  future  force  success  and  survivability  in  combat.  The  desired 
voice  response  from  a  computer  should  contain  information  which  ensures  the  sender  that  the  receiver 
correctly  heard  the  information  and  is  going  to  provide  the  reaction  desired  by  the  sender.  The 
communications  from  the  computer  must  address  which  vocal  information  it  is  responding  to  and, 
in  action  summary  form,  inform  sender  of  action  to  be  taken.  System  must  react  to  a  "NEGATIVE" 
t3q)e  statement  to  halt  a  reaction  which  was  misinterpreted  or  to  carry  on  with  the  acceptance  of  a 
default  response  statement  similar  to  "ROGER  "  or  "OK". 
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The  virtual  crew  member  should  be  the  central  processing  system  for  computer  generated  cautions, 
warnings,  and  advisories  (CWA).  CWAs  are  ever  present  operational,  administrative  and  especially 
in  training  environments.  However,  interactive  voice  CWAs,  which  can  answer  analytical  questions 
you  pose  after  the  initial  warning  are  required  with  emerging  technologies.  If  you  receive  a  low  fuel 
warning,  it  would  be  logical  to  ask  and  receive  response  to  how  much  is  left,  estimated  distance 
supported,  is  consumption  above  normal  level  possibly  indicating  a  leak,  etc.  CWAs  must  become 
interactive  and  analytical  as  they  are  included  in  future  technologies  to  ensure  success  of  future  force 
projection  and  survivability  during  combat.  The  inherent  ability  of  future  systems  must  include  those 
capable  of  performing  an  immediate  logical  analysis  of  CWAs,  not  only  for  verification,  but  to  act 
immediately  from  multiple  sensor  input  or  to  provide  an  immediate  impact  assessment  and  provide 
options  available  to  the  crew. 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Mounted  Warfighting  Battlespace 


Title:  Virtual  Crewmembers 

Concept:  In  training/simulation,  a  platform  member  works  w/virtual  crewmembers  to 
carry  out  an  objective. 

Technical  Assessment:  (Desired  Capabilities,  Current  Shortfalls) 


Warfighting  Merit:  (Value  added  to  Soldier) 
—  Low  cost  traimng 


Current  Projects: 

Required  Research: 

—  Context  study  (What  do  crewmembers  say?  How  do  they  interact?) 

—  Interface  artificial  agents  (expert  systems)  w/training  simulator 

—  Support  necessary  NLP 
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NUMBER:  8 


SPOKEN  HUMAN-MACHINE  DIALOGUE  WORKSHOP 

PANEL:  MOUNTED  WARFIGHTING  BATTLESPACE. 

EVALUATION  AREA:  ASSISTANCE. 

TITLE;  CREW  STATION  ENHANCEMENT 

ANALYST:  CHUCK  CAMPBELL  &  MARTIN  BOSEMER 

MBBL,  FORT  KNOX,  KY  40121  DSN:  464-1963/2045 

WARFIGHTING  MERIT  (VALUE  ADDED  TO  SOLDIER) 

Crew  station  enhancements  will  be  inherent  with  the  incorporation  of  spoken  human  machine 
dialogue.  Any  time  you  can  minimize  the  time  a  crew  member  has  to  move  eyes  from  the  field  or 
function  being  performed,  especially  during  combat,  you  increase  the  effectiveness  and  survivability 
of  the  platform  crew.  Within  a  tank  system,  being  able  to  verbally  have  the  autoloader  switch  from 
sabot  to  heat  and  the  switch  at  the  gunners  station  automatically  switch  from  sabot  to  heat,  and  the 
sighting  systems  automatically  adjust,  will  allow  the  gunner  and  tank  commander  to  concentrate  on 
the  target(s),  vice  requiring  thought  and  movement  to  switch.  This  incorporates  the  virtual  crew 
member  as  an  absolute  requirement  which  we  are  slowly  evolving  towards  with  today's  technology. 
Technology  has  provided  for  many  sophisticated  independent  systems  which  are  evolving  from  fixed 
program  based  systems  to  developed  logic  based  systems  which  approach  artificial  intelligence. 
Crew  station  enhancement  is  the  virtual  crew  member,  one  that  appropriately  responds  with  the 
correct  action/reaction  within  less  time,  at  greater  efficiency,  and  at  a  level  exceeding  a  well  trained 
and  seasoned  veteran  soldier  while  interfacing  well  with  other  crew  members.  The  virtual  crew 
member  should  also  be  a  single  source  of  information  integration  from  independent  platform  systems. 
Every  system  should  operate  independently  so  damage  to  one  system  will  have  a  minimum  impact 
on  other  systems.  The  virtual  crew  member  should  not  only  perform  certain  functions,  such  as  the 
loaders  position  on  a  tank  (intelligent  autoloader),  but  should  also  be  able  to  integrate  information 
exchange  between  all  platform  systems. 

The  ability  to  quickly  evaluate  the  functional  status  of  platform  systems  and  to  quickly  attain 
reference  material  will  be  of  immeasurable  assistance  to  platform  crew  members  as  they  have  to 
perform  field  expedient  repairs.  The  computer  can  in  fact  lead  them  step  by  step  through  the  problem 
analysis  and  identification  phase  on  through  the  actual  repair  phase.  This  ability,  based  on 
independent  systems  information  connectivity,  will  enhance  the  operational  capability,  survivability, 
and  sustainability  of  any  mounted  platform.  This  capability  vvill  be  of  immense  importance  to  any 
crew  member  at  any  station. 

Spoken  human  machine  dialogue  capability  will,  in  it's  self,  improve  crew  station  effectiveness  since 
the  soldier  is  accustomed  to  initiating  and  responding  to  verbal  communications  more  so  than  any 
other  form  of  communication.  The  ability  of  a  computer  system  to  respond  to  voice  communications 
is  extremely  important  and  critical  to  future  force  success  and  survivability  in  combat.  The  greatest 


crew  station  enhancement  is  the  analytical  ability  afforded  to  the  crew  members  by  a  "virtual  crew 
member". 

The  virtual  crew  member  should  not  only  be  the  central  processing  system  for  performing  explicit 
functions  normally  performed  by  a  crew  member.  It  must  become  the  interactive  and  analytical 
interface  between  all  platform  systems  and  the  platform  crew  to  ensure  success  of  future  force 
projection  and  survivability  during  combat.  The  inherent  ability  of  future  systems  must  include  those 
capable  of  performing  an  immediate  logical  analysis  of  cautions,  warnings,  and  alarms,  not  only  for 
verification,  but  to  act  immediately  from  multiple  sensor  input  or  to  provide  an  immediate  impact 
assessment  and  provide  options  available  to  the  crew. 

Crew  station  enhancements  made  available  through  spoken  human  machine  interface  are  composite 
enhancements.  Many  improvements  will  occur,  at  various  times,  improving  the  environment  of  the 
crew  member.  We  must  accept  the  fact  that  in  today's  Army,  the  size  of  crews  are  becoming  smaller 
while  the  platform  capability  and  technology  continues  to  increase.  This  is  putting  an  increased 
broad  and  in-depth  technological  knowledge  requirement  on  fewer  crew  members,  requiring  faster 
actions/reactions  of  those  members.  The  future  effectiveness  and  survivability  of  future  platforms 
will  require  effective  interaction  between  man  and  machine.  Spoken  human  machine  dialogue  is  the 
next  step  in  technology  required  towards  direct  mental  human  machine  interface. 
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NUMBER:  9 


SPOKEN  HUMAN-MACHINE  DIALOGUE  WORKSHOP 

PANEL:  MOUNTED  WARFIGHTING  BATTLESPACE. 

EVALUATION  AREA:  ASSISTANCE. 

TITLE:  DICTATION  OF  PLANS,  ORDERS,  AND  DIRECTIVES 

ANALYST:  CHUCK  CAMPBELL  &  MARTIN  BO SEMER 

MBBL,  FORT  KNOX,  KY  40121  DSN:  464-1963/2045 

WARFIGHTING  MERIT  (VALUE  ADDED  TO  SOLDIER) 

The  volume  of  communications,  information,  and  data  is  increasing  at  an  exponential  rate.  The  US. 
has  witnessed  an  unprecedented  growth  in  data  processing  over  the  past  twenty  years.  Managers  no 
longer  have  the  ability  to  review  transaction  ledgers,  only  summaries.  We  have  evolved  into  a 
"BOTTOM  LINE  MANAGEMENT"  age  due  to  the  ever  increasing  complexity  and  volume  of 
information.  With  ever  increasing  volumes  and  types  if  information/data  sources,  computer  systems 
commonly  assimilate  information  into  short  answers.  Communication  message  extraction,  gisting, 
and  screening  is  predominantly  accomplished  by  administrative,  operational,  and  staff  persormel  on 
a  daily  basis.  In  almost  every  environment,  topics  and  bullet/paragraph  supporting  information  is  used 
when  higher  levels  of  management  are  being  informed.  Effective  and  concise  communication  message 
extraction,  gisting,  and  screening  will  be  required  from  spoken  human  machine  dialogue  and  advanced 
artificial  intelligence  systems  in  future  warfighting  situations  and  mounted  platform  operations  for 
successful  combat/operational  missions. 

Spoken  human  machine  dialogue  is  a  natural  progression  from  keyboard  entry  systems.  It  will  be  a 
natural  interaction  to  obtain  information  required  for  the  preponderance  of  administrative  tasks  and 
to  produce  hard  copy  output.  In  future  technology,  written  correspondence  with  signatures  for 
authentication  will  be  replaced  by  spoken  human  machine  dialogue  using  a  voice  print  for  a  higher 
level  of  verification/authentication.  This  technology  will  revolutionize  the  development/issuance 
process  for  plans,  orders,  and  directives  .  If  the  current  process  is  not  dramatically  improved, 
current  methods  will  rapidly  become  too  slow  for  processing  the  flow  if  information/data  which  will 
have  a  devastating  impact  on  daily  operations  and  in  combat. 

Spoken  human  machine  dialogue  must  revolutionize  current  concepts  of  effective  command  systems. 
As  we  move  further  toward  a  paperless  environment  we  must  also  move  towards  a  keyless 
environment.  Spoken  human  machine  dialogue  is  just  the  next  logical  step  required  towards  direct 
mental  interface  between  humans  and  machines.  Our  views  of  current  methods  of  manually 
developing  plans,  orders,  and  directives  must  be  replaced  with  the  vision  of  a  small  wireless 
headset/ear  device  and  microphone,  which  are  capable  of  independently  determining  which  voice 
data  is  intended  for  transmission,  destination(s)  desired,  additional  data  to  be  attached  to 
transmission,  most  effective  and  economical  method  of  transmission,  security/encryption  levels 
required,  interpretation  of  information  originating  in  a  language  other  than  the  users,  and  automatic 
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switching/multi-charmel  simultaneous  routing  of  information.  These  systems  must  also  be  able  to 
process  spoken  input  as  visual  output  in  multiple  formats  such  as  character,  graphic,  digital,  etc. 

The  financial,  legal,  and  proof  positive  requirements  have  generated  an  immense  requirement  for 
every  profession  to  produce  and  maintain  large  volumes  of  "hard  copy  documentation".  Instilling 
a  sociological  reform  towards  "hard  copy  documentation"  will  be  the  greatest  challenge  in  the 
emergence  of  this  technology  within  command  and  related  administrative  areas.  As  an  example,  the 
technology  of  manually  typed  transcription/recording  services,  commonly  used  in  court  rooms,  are 
being  parallel  with  court  cameras  and  recording  devices,  but  not  replaced.  Courts  still  require  court 
recorders.  Until  voice  recognition  systems,  high  volume  data  storage,  enhanced  information  retrieval 
systems,  and  of  most  importance  relational  network  interface  systems  for  information  sharing/routing 
are  perfected  and  accepted  as  an  "OFFICIAL  RECORD",  benefits  will  be  comparatively  minimum. 
Transcription  of  plans,  orders,  and  directives  may  occur  through  spoken  human  machine  interface, 
buy  technologically  speaking,  this  will  only  be  a  minor  improvement  when  one  envisions  the  total 
process. 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Mounted  Warfighting  Battlesoace 

Assistance  Operations 

(Circle  One) 

Title:  Mission  Training  with  Simulated  Forces. 

Concept:  When  semi-automated  (simulated)  forces  are  an  important  part  of  a  mission 
training  exercise,  the  need  for  realism  is  important.  The  ofiBcer  being  trained  (for  example, 
a  battalion  commander)  should  be  able  to  communicate  with  simulated  subordinates  in  the 
same  way  he  would  communicate  with  real  subordinates.  This  means  human-machine 
dialog  over  a  simulated  radio. 

Technical  Assessment:  (Desired  Capabilities,  Current  Shortfalls) 

Desired  capabilities  include  the  ability  of  semi-automated  forces  (SAFOR)  to  understand 
the  small  subset  of  natural  language  that  is  relevant  to  military  commanders  sending  short 
orders  to  subordinates;  the  ability  of  SAFOR  to  generate  synthesized  voice  output;  the 
ability  (for  the  sake  of  realistic  training)  to  generate  simulated  radio  static  and  crosstalk; 
the  ability  to  recognize  voice  utterances  in  spite  of  static  and  crosstalk. 

Warfighting  Merit:  (Value  added  to  Soldier) 

The  value  is,  first,  increasing  the  level  of  realism  when  training  with  SAFOR.  Secondly, 
voice  communication  between  the  training  audience  and  the  SAFOR  will  reduce  the  need 
for  expensive  support  personnel  to  make  SAFOR  simulations  run  property. 

Current  Projects: 

In  the  Modular  Semi-Automated  Forces  (MODSAF)  simulation  being  developed  by 
ARP  A  (armored  battalion  level  training),  an  experiment  with  voice  communication  is 
being  conducted  by  STRICOM.  The  work  is  bring  done  at  MITRE. 


Required  Research: 

Robust  voice  recognition  (especially  with  respect  to  background  noise)  is  essential. 
Second,  currently  available  GTS  voice  recognition  systems  often  do  not  make  available  in 
their  output  enough  information  to  enable  Natural  Language  Processing  to  detect  and 
correct  single-word  recognition  errors.  Domg  this  is  currently  feasible,  but  pressure  needs 
to  be  applied  to  GTS  vendors  to  get  them  to  do  it.  Third,  research  is  needed  to  clasrify 
the  kinds  of  dialogs  that  commanders  and  their  subordinates  actually  have  over  real 
commo  channels  (currently  radio),  so  that  SAFOR  can  take  part  in  such  dialogs. 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Mounted  Warfighting  Battlesoace 


Training 


Assistance 
(Circle  One) 


Operations 


Tide:  Integration  of  training,  assistance,  and  operational  spoken  human-machine 
interfaces. 

Concept:  Develop  and  integrate  natural  language  program  so  that  the  same  program  (or 
at  least  same  interface)  is  used  in  both  training  of  crewmen  and  maintenance  personnel  and 
would  act  as  an  assistance  in  the  field  and  could  be  accessed  at  C&C  level  for  operational 
design. 

Technical  Assessment;  (Desired  Capabilities,  Current  Shortfalls) 

Current  interface  are  all  designed  for  a  specific  function/application.  Since  the  same 
knowledge  is  required  for  levels  (only  how  it  interfaces)  with  the  individual  charges. 
Natural  language  processing  would  be  a  key  component  in  the  interface,  especially  as  it 
can  easily  transcend  different  platform  (or  interfiice  level). . 

Warfighting  Merit:  (Value  added  to  Soldier) 

Would  promote  commonality  access  training  and  deployment.  Should  reduce  over^ 
development  cost  since  it  would  eliminate  duplication  of  effort  required  for  developing  3 
different  systems. 


Current  Projects: 
Unknown. 


Required  Research:  - 

Extremely  high  level  at  integration  is  required  not  only  to  make  sure  that  is  combines  all 
the  interfaces  into  a  unified  product  but  also  integrates  correctly  with  other  programs  the 
system  must  work  with  at  each  level  of  deployment. 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Mounted  Warfiehtine  Battlespace 

Assistance  Operations 

(Circle  One) 

Title:  Automated  evaluator  for  training  system  speech  recognition  -  complex 
tasks 

Concept:  Use  of  speech  recognition  to  score  students  or  simulation  task  performance  in 
place  of  human  evaluator  to  automate  evaluation  process.  For  example,  student  needs  to 
make  site  report.  Use  speech  recognition  to  evaluate  accuracy  of  site  report. 

Technical  Assessment:  (Desired  Capabilities,  Current  Shortfalls) 

High  accuracy,  continuous  speech  recognition  technology. 

Warfighting  Merit:  (Value  added  to  Soldier) 

Reduced  personnel  for  evaluating  students 

Current  Projects: 

Air  Force  program  called  SECT  -  Student  Electronic  Combat  Trainer  is  using  ITT  speech 
recognition  to  evaluate  students  during  EW  training. 

Required  Research: 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Mounted  Warfiphting  Battlesoace 

Assistance  Operations 

(Circle  One) 


Title:  Tutoring  Maintenance 

Concept:  Student  can  speak  to  “articulate  expert”  in  a  VR  simulation  for  tank 

maintenance.  Student  poses  questions  (“Why  x?”)  and  teaching  requests  (“Show  me  X”) 
and  expert  responds. 

Technical  Assessment:  (Desired  Capabilities,  Current  Shortfalls) 

Continuous  speech  recognition  of  limited  domain  questions,  natural  or  open  syntax  (as  in 
Air  Travel  Information  System  Task).  Use  topic  spotting  of  natural  language  processing 

to  aid  mterpretation  of  questions.  Current  technology  does  not  mtegrate  NLP  and  higher 
knowledge  with  CSR.  ^ 

Warfighting  Merit:  (Value  added  to  Soldier) 

--  ^proved  trai^g  of  tank  (and  other  ground  platform),  maintenance  for  tank  and 

platform  crew  will  reduce  reliance  on  logistics  base,  so  increase  time  task  available  for 
battle  operations. 

—  Architecture  can  be  reversed  for  other  kinds  of  tutors. 

Current  Projects: 

—  Duke  University  tank  maintenance  incorporates  VR  and  tutor  architecture 

—  MITRE  experiments  on  NLP-CSR  integration  (also  SRI,  BBN). 

—  CSR  reswch  push  beyond  current  COTS  level  (eg  CHU,  SRI). 

~  BBN  articulate  experts  for  maintenance  of  Hawk  ADS. 

Required  Research: 

—  Model  dialogues  between  real  students  and  tutors. 

—  Develop  NLP/dialogue  planning/processing  modules 

—  Develop  expert  knowledge  base 

—  Refine  CSR  technology  and  integrate  NLP. 

~  Integrate  with  articulate  expert  and  VR  simulation 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Mounted  Warfightine  Battlesoace 

Training  Assistance 

(Circle  One) 

Title:  Real  time  information  management  through  voice  query 

Concept:  Use  of  voice  recognition  to  rapidly  query  on-board  (battle)  mission 
managementfinformation  management  computer  system  to  retrieve  mission  critical 
information.  Intelligent  filtering  of  data  into  useful  information,  i.e.  threat  position. 

Technical  Assessment:  (Desired  Capabilities,  Current  Shortfalls) 

Natural  Language  data  base  query  methods.  High  accuracy,  robust  speech  recognition  in 
noise,  vibration  environment. 

Warflghting  Merit:  (Value  added  to  Soldier) 

Improved  situation  awareness,  increased  surviability. 


Current  Projects: 


Required  Research: 

Basic  research  in  improving  speech  recognition  in  severe  environments. 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Mounted  Warfightine  Battlesoace 


Training  Assistance 

(Circle  One) 

Title:  Command  and  control  on  the  move 

Concept:  Allow  full,  clear  and  distinct  use  of  command  and  control  systems  while 
operating  on  the  move.  Must  operate  within  the  platform  environment. 

Technical  Assessment:  (Desired  Capabilities,  Current  Shortfalls) 

Must  filter  any  background  noise,  vibration,  shock  of  weapons  firing  and  other  system 
operating  distracters  while  crew  is  operating  under  stress. 

Warflghting  Merit:  (Value  added  to  Soldier) 

Allows  rapid,  accurate  transmission  of  command  and  control  plans,  orders  and  directives. 
Current  Projects: 

Required  Research: 

Ability  to  have  speech  recognition  with  built  in  responses  that  commands  are  understood. 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Mounted  Warfighting  Battlespace 

Training  Assistance 

(Circle  One) 


Title:  Control  of  robotics 

Concept:  Voice  actuated  command  and  control  of  robotics  platforms,  air  and  ground,  in 
reconnaissance  or  mine/obstacle  clearance. 

Technical  Assessment:  (Desired  Capabilities,  Current  Shortfalls) 

Ability  to  voice  control  a  platform  from  a  remote,  semi-secure  position.  Systems  are  now 
teleoperated  and  have  very  cumbersome,  resource  demanding  control  system. 

Warflghting  Merit:  (Value  added  to  Soldier) 

Can  save  soldier  lives.  Permits  better  utilization  of  resources,  assists  in  the  maintenance  of 
battle  tempo. 


Current  Projects: 


Required  Research: 

Voice  activation  of  systems  that  will  allow  navigation  and  control  of  platform  systems. 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel: 

Trainin 


s 


Title:  Cautions,  Warnings  and  Advisories  (relevant  information  pertaining  to  internal 
and  external  imminent  threat) 


Concept:  Improve  platform  caution,  warning  and  advisory  alerting  system  through 
intelligent  application  of  voice  warning  techniques  in  place  of  tones,  and  lights. 


Technical  Assessment:  (Desired  Capabilities,  Current  Shortfalls) 

Use  of  voice  output  for  CWA  must  be  intelligently  applied  to  minimize  false  alarms  and  to 
not  compete  with  other  tasks  requiring  high  aural  attention  unless  the  warning  is 
immediate  and  potentially  life  threatening.  Avoid  the  “Bitchin  Betty”  implementation. 

Warfighting  Merit:  (Value  added  to  Soldier) 

Enhanced  crew  awareness  of  threat  warnings  and  system  malfunctions. 


Current  Projects: 


Required  Research:  Development  of  intelligent  prioritization  schemes  for  voice 
communication  of  cautions,  warnings,  and  advisories. 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Mounted  Warfighting  Battlesoace 

Trainine  x'^sistance  N.  Operations 

^  (Circle  One)  ) 

Title:  Dictation  of  plans,  orders  and  directives 

Concept:  Commander  voice  dictates  orders,  plans,  directives  into  system.  System 
formats,  adds  graphics  and  prepares  to  transmit  digitally. 

Technical  Assessment:  (Desired  Capabilities,  Current  Shortfalls) 

Ability  to  prepare/format  orders,  plans  and  directives. 

Warflghting  Merit:  (Value  added  to  Soldier) 

Maintains  battle  tempo. 


Current  Projects: 


Required  Research: 

Requires  speech  recognition;  natural  language  processor. 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Mounted  Warfiehting  Battlespace 

Training  x^X^istance  ^  Operations 

(  (Circle  One)  ) 


Tide:  Crew  Station  Enhancement:  Extended  Reach  of  the  Crewmember 

Concept:  Give  crew  members  “infinite  reach”  through  spoken  natural  language.  Can 
access  equipment,  information  w/out  physically  moving  to  location.  Warnings,  advisories 
spoken. . 

Technical  Assessment:  (Desired  Capabilities,  Current  Shortfalls) 

Spoken  language  interface  to  platform  systems 
Shortfalls  —  many  platform  fonctions  not  electronicaUy  accessible 
~  fairly  broad  vocabulary 

Warfighting  Merit:  (Value  added  to  Soldier) 

Crew  members  can  do  more,  in  less  time,  simultaneously. 


Current  Projects: 

Required  Research: 

Must  be  extremely  reliable. 
Handle  all  sorts  of  noise. 
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COMBAT  SERVICE  SUPPORT 


css  Applications  of  SHMD 

The  CSS  panel  felt  that  the  application  of  SHMD  should  be  viewed  more  as  a  continuum  between 
training,  assistance,  and  operations  vice  distinct  areas.  The  primary  rationale  for  this  viewpoint  is 
that  we  should  train  as  we  fight.  For  the  purpose  of  this  summary  the  potential  applications  are 
discussed  separately. 

Training  Applications; 

The  CSS  panel  discussed  five  training  applications  of  SHMD;  (1)  learning  the  Army  language,  (2) 
sustainment  training,  (3)  training  on  the  use  of  expert  systems,  (4)  load  plan  training,  and  (5)  natural 
language  for  distributed  interactive  simulations. 

SHMD  could  be  used  to  teach  entry  level  soldiers  the  Army  s  language.  This  would  include  learning 
the  mission  specific  language,  acronyms,  and  vocabulary.  Such  a  system  would  decrease  the  time 
it  takes  to  integrate  a  new  soldier  into  the  Army. 

A  vmce  recognition,  interactive  system  could  be  used  for  sustainment  training  for  various  Standard 
Army  Management  Infomution  System  (STAMIS).  The  panel  recommended  starting  with  the 
Standard  Army  Retail  Supply  System  -  Objective  (SARSS-0). 

The  third  trair^  application  is  to  use  SHMD  do  supplemental  training  in  the  use  of  expert  systms 
by  using  voice  iiqxit  and  output.  This  would  permit  a  rapid  training  on  the  use  of  the  expert  system. 

SHMD  could  be  used  to  training  the  CSS  community  on  how  to  load  ships,  rail  cars,  and  planes 
without  wasting  space.  The  current  system  for  training  doesn  t  provide  the  feedback  to  the  soldier 
on  how  well  the  space  was  used.  Such  as  ^stem  would  permit  faster  more  efficient  stow  plans. 

The  final  training  application  was  to  use  natural  language  inputs  and  outputs  with  distributed 
interactive  simulations.  The  concept  would  be  for  the  ^stem  to  provide  information  on  the  location 
of  the  closest  supply  point  or  maintenance  facility  during  a  distributed  interactive  simulation 
exercises. 

Assistance  Applications; 

The  CSS  panel  came  up  with  seven  assistance  applications;  (1)  integrated  electronic  technical 
manuals,  (2)  field  services  (mortuary  affairs),  (3)  back  seat  driver,  (4)  Explosive  Ordinance  Disposal 
(EOD)  disarming,  (5)  Standard  Army  Management  Information  System  (STAMIS),  (6)  voice  print 
for  supply  distribution,  and  (7)  natural  language  for  doctors. 

SHMD  could  be  used  to  replace  technical  manuals  with  wearable  computers  and  use  natural 
language  as  an  input  device  which  is  augments  virtual  reality  with  voice.  Such  a  system  could 
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diiranate  the  second  soldier  who  now  reads  the  technical  manual  to  a  soldier  who  is  inside  the  tank. 
It  would  also  reduce  the  space  and  weight  of  paper  technical  manuals.  It  could  also  reduce  the 
training  requirements  and  the  IQ  requirements  for  maintainers. 

A  voice  activated  system  that  would  provide  latitude  and  longitude  data  for  the  location  and 
operational  procedures  when  recovering  remains  was  one  of  the  assistance  uses  for  SHMD. 

The  back  seat  driver  would  provide  assistance  to  the  driver  on  navigation,  fuel  required  to  get  to  a 
certain  location  and  driving  in  blackout  conditions. 


The  EOD  disarming  would  use  a  intelligent  knowledge  based  system  to  assist  in  bomb  disposal 
training  and  disarmament  for  conventional  ordinance. 

A  interactive  systems  starting  at  the  unit  level  through  the  division  into  corps  would  provide 
diagnostic  procedures  indicating  the  status  as  well  as  evacuation  procedures  through  maintenance 
activities  as  well  as  repair  part  requisitioning  to  include  voice  query. 

A  voice  print  could  be  used  to  provide  verification  of  individual  receiving  supplies.  This  would 
permit  the  rapid  receipt  of  supplies. 


Natural  language  could  be  used  to  assist  medical  doctors  either  during  an  operation  or  in  diagnosis 
of  an  illness.  Dictation  could  be  used  to  input  to  medical  records. 

Operational  Applications: 

The  CSS  panel  come  up  with  three  operational  applications:  (1)  Nuclear,  Biological  and  Chemical 
(NBC)  detection,  (2)  voice  activated  radio  fi'equency  (RF)  tags,  (3)  environmental  warfighting  suit. 

A  voice  capability  that  could  be  added  to  the  Chemical  Detect  alarm  system  that  will  tell  the  user 
what  agent  was  detected  was  identified  as  a  needed  capability. 

Another  operational  application  of  SHMD  is  for  a  voice  activated  RF  tag.  This  tag  would  provide 
voice  re^XMise  to  spoken  queries  concerning  container  contents  in  include  consignor  and  consignee. 

The  final  operational  application  was  for  an  environmental  warfighting  suit.  The  uniform  would 
consist  of  sensors  and  environmental  awareness  data  with  voice  query  and  r^x}gnition. 
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css  Application  of  SHMD 


CSS 

Applieatiofw 

of 

Spokwi  Human*  maetiino  Dialogw 


SHMD  CSS  Rsiatlonthipt 


LtamliHi  Hm  Army  tangiMg* 

•  Mission  spacNie  language 

•  Acronyms 

•  Vocabulaiy 


Retsarch  Artat 

•  Speaker  Independent 

•  Noisy  environments 

•  Statistical  models  of  language  Including 
dialogs 

•  Integration  of  structural  information  in 
language 

•  Multilingual  systems 

Bottom  Ine  •  Need  to  move  out 
and  use  SHMD  nowl 


Training  Applicatlona 

1.  Leamkigttia  Annylanguaga. 

2.  Sustainment  trainkig. 

3.  Training  on  use  of  expert  systems. 

4.  Load  Plan  Training. 

5.  NL  for  Distributed  Interactive  Simulation. 


•  Input/output 


•  Maintab)  skill  level 


6/1/95 
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css  Application  of  SHMD 


Training  on  use  of 
Expert  Systems 

•  Supplement  with  voice 


NL  input  and  output  into 
Distributed  Interactive  Simulation 
(DIS) 

•  Find  supply  point 

•  Find  maintenance  faciltti^ 

•  Ammo  and  fuel  updates 


Integrated  Electronic 
Technical  Manuals 

•  Wearable  computer 

•  Augment  virtual  reality  with  voice 

•  Reduce  f»per  manuals 


Load  Plan  Training 

*  Current  models  are  for  operation 

•  Feedback  on  wasted  s|Mice 


Assistance  Applications 

1.  Integrated  Electronic  Tedi  Manuals 

2.  Field  ServicM  (Mortuary  Affaks) 

3.  Back  Seat  Driver 

4.  EOO  disposal 

5.  STAMIS 

6.  Voice  print  for  supply  dIstrttMition 

7.  NL  for  doctors 


Field  Services  (Mortuary  Affairs) 

•  Position  locaflond  remains 

•  Procedures  for  recoverliHjremaina 


5/31/95 
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css  Application  of  SHMD 


Back  Seat  Driver 

•  Assistance  to  find  location 

•  Night  driving  (blackout) 


Voice  Input  to  STAMIS 

■  Unit  through  Corps 

•  Diagnostic  procedures 

•  Maintenance  procedures 

•  Requisitioning  procedures 

NL  Assistance  to  Doctors 

•  During  opwations 

•  Making  diagnoeis 


5/31/96 


Explosive  Ordnance 
Disposal  (EOD)  disarming 

•  Bomb  disposal 

•  Disarmament  of  conventional  ordnance 


Voica  print  for  supply  distribution 

•  Voice  ID  for  receiving  supplies 

•  Need  major  breakthrough 


Operational  Applications 

1.  NBC  detection 

2.  Voice  activated  RF  tags 

3.  Environmental  Warfighting  SuR 
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css  Application  of  SHMD 


Voice  Activated  RF  Tags 


NBC  Detection 


•  Container  contents 


*  Voice  alarm 

<  Integrate  detection,  analyst  and  alarm 


Environmental  Warfighting  Suit 

•  Every  soldier  a  sensor 

•  Voice  dialog 


5/31/95 
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Required  Research 


(Preamble) 

Research  in  the  past  10  -IS  years  haye  led  to  several  technologies  in  Speech  and 
Language  that  are  now  ready  to  be  deployed  in  many  application  areas  for  the  Army  -  in 
training,  assistance,  and  operational. 

What  are  the  areas  of  research  that  need  further  development  in  the  next  10  - 15 
years  which  will  lead  to  new  technologies  that  will  only  improve  on  the  applications 
described  earlier  but  open  up  new  applications. 

*  Further  woric  on  speaker  independent  ^sterns 

*  Speech  systems  that  work  effectively  in  very  noisy  environments 

*  Statistical  models  of  language  including  (halogs 

*  Integration  of  statistical  structured  information  in  language 

*  Multilingual  systems  for  multilingual  environments 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Combat  Service  Support 
(Training) 

Title:  Sustainment  for  Standard  Army  Retail  Supply  System  -  Objective  (SARSS-0) 


Concept:  Voice  recognition  interactive  system  for  sustainment  training-  For  logistics 
within  theater  through  Depot  to  include  S AVI  Tag  Recognition 

Technical  Assessment: 

Warfiahting  Merit:  Provide  method  to  maintain  skill  level  on  Standard  Army 
Management  Information  Systems  (STAMIS) 

Current  Projects: 


Required  Research: 

-  Ambient  Noise  resistant  systems 

-  Speaker  independent  ^sterns 


POC  Karlo  Aguilar 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Combat  Service  Support 
(Training) 

Title:  NL  input  and  output  into  Distributed  Interactive  Simulation  (OIS) 

Concent:  Have  the  system  provide  information  on  location  of  closest  supply  point  or 
maintenance  facility  during  a  DIS  training  exercise.  Could  also  provide  ammo  and  fuel 
updates. 


T«»fhnigal  Assessment:  Would  like  to  be  able  to  ask  the  computer  w^e  the  closest 
supply  point  is,  etc.  Computer  will  analyze  system  messages  and  provide  answer. 


Warfighting  Merit:  To  be  used  in  DIS  trmning  exercises  using  virtual  reality. 


Current  Projects:  STOWE,  SIMNET 


Required  Research:  Dialog  systems 


POC  Miranda  Moore 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Combat  Service  Support 
(Training) 

Title:  Learning  the  Army  language 


Concent:  Use  NL  to  teach  entry  level  soldiers  the  Army’s  language.  This  would  include 
learning  the  nnssion  specific  language,  acronyms,  and  vocabulary. 


Technical  Assessment: 


Warfighting  Merit:  Would  decrease  the  time  it  takes  to  integrate  a  new  soldier  into  the 
Army. 

Current  Projects: 

None 

Required  Research: 

Speaker  independent  systems 


POC 


Mary  Lee  Johnson 


Spoken  Human-Machine  Dialogue  Workshop 

Panel:  Combat  Service  Support 
(Training) 

Title:  Training  on  the  use  of  Expert  systems 

Concent:  Training  on  how  to  use  expert  systems  should  be  supplemented  with  voice 
input  and  output. 

T»«^hni<;»ai  Assessment:  Replace  written  dialogue  with  voice. 

Warfighting  Merit:  More  efiBcient  and  eaaer  to  understand  and  would  quickly  train  the 
user  on  the  expert  system. 

Current  Projects: 

Required  Research: 

Dialog  ^sterns 


POC  Miranda  Moore 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Combat  Service  Support 
(Training) 

Title:  Load  Plan  Training 

Concent:  Natural  language  providing  feedback  on  wasted  space  on  a  ship/railcar/plane 
etc.  to  train  to  more  efficiently  load  a  ship. 


Technical  Assessment:  Ship  stowplanning  systems  today  do  not  train  to  load  more 
efficiently,  they  provide  an  automated  capability  to  build  a  stow  plan.  Assessment  of  stow 
plan  would  be  useful  for  training  to  load  a  ship. 


Warfightina  Merit:  Faster  more  efficient  ship  stow  plans. 


Current  Projects:  CAEMS,  ICODES 


Required  Research:  Interfacing  optimization  or  simulation  routines  that  can  analyze 
stow  plans  to  speech  language  systems. 


POC  Miranda  Moore 
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Spoken  Human*Machine  Dialogue  Workshop 


Panel:  Combat  Service  Support 
(Assistance) 

Title:  Integrated  Electronic  Tech  Manuals  (lETM) 

Coneent:  Replace  paper  Tech  manuals  with  wearable  computers  and  use  ML  as  an  input 
device.  Augment  virtual  reality  vith  voice. 

j^hnicai  Assessment:  Speech  recognition  and  understanding  needs  to  improve.  Must 
approach  that  of  humans. 


Warfighting  Merit: 

1 .  Eliminates  the  second  soldier  who  now  reads  the  Techmcal  manual  to  the  hands 
on  soldier  who  is  inside  a  tank. 

2.  Reduce  space  and  weight  of  paper  Technical  manuals. 

3.  Reduce  the  training  requirement  and  the  IQ  requirement. 

Current  Proi^ts:  lETM  for  Paladin/AFAS/Crusader 


Required  Research: 

-  Ambient  noise  renstant  systems 
.  Dialog  ^sterns 


POC  Edward  Grosch 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Combat  Service  Support 
(Assistance) 

Title:  Field  services  (Mortuary  affairs) 

Concent:  Voice  activated  system  that  would  provide  Latitude/Longitude  data  for 
location  together  with  specific  operational  procedures  when  recovering  remains. 


Technical  Assessment: 


Warfighting  Merit:  Efficient  recovery  operations 


Current  Projects: 
None 

Required  Research: 


POC  Karlo  Aguilar 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Combat  Service  Support 
(Assistance) 

Title.  Back  seat  driver 

Concent:  Provide  assistance  to  the  driver.  Using  voice  the  driver  could  ask  for 
assistance,  for  example,  like  how  to  get  to  a  location,  how  much  gas  is  left  or  required  to 
get  to  a  certain  location.  Night  driving  in  blackout  conditions. 

Technical  Assessment. 


Warfighting  Merit:  Link  to  GPS  so  that  the  soldier  knows  where  he  is  at  all  times.  And 
route  to  take  to  get  to  location. 


Current  Projects: 

None 

Required  Research: 

-  Ambient  noise  resistance  ^sterns 
•  Dialog  systems 


POC  JefifJahnke 
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Spoken  Human-Machine  Dialogue  Workshop 

Panel:  Combat  Service  Support 
(Assistance) 

Title:  Explosive  Ordinance  Disposal  (EOD)  disarming 

Concept:  Use  intelligent  knowledge  based  system  to  assist  in  bomb  disposal  training  and 
disarmament  for  conventional  ordinance. 


Technical  Assessment:  Step  by  step  procedure  to  disarm  conventional  ordnance. 
System  can  query  a  manual  built  into  the  database  to  step  by  stq>. 


Warfiahting  Merit:  Hands  free  to  do  work 


Current  Projects: 


Required  Research: 


POC  Jeff  Jahnke 
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Spoken  Humin-Machine  Dialogue  Workshop 

Panel:  Combat  Service  Support 
(Assistance) 

Title:  Voice  input  to  Standard  Army  Management  Information  System  (ST AMIS) 

Concent:  Interactive  ^ems  starting  at  the  unit  level  through  the  division  into  corps  - 
the  Unit  Level  Logistics  System  (ULLS)  system  would  provide  diagnostic  procedures 
indicating  status  as  well  as  evacuation  procures  through  maintenance  activities  as  well 
as  repair  part  requisitioning  to  include  voice  query. 


Technical  Assessment: 


Warfighting  Merit:  Enhanced  battlefield  logistics  support  operational  status 


Current  Projects: 


Required  Research: 


POC  Karlo  Aguilar 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Combat  Service  Support 
(Assistance) 

Title:  Voice  print  for  supply  distribution 

Concent:  Use  NL  to  provide  verification  of  individual  ra:eiving  supplies. 


Technical  Assessment:  Current  shortcoming  is  the  ability  of  individuals  to  duplicate 
voices  or  q)eech  patterns  of  others;  the  different  print  produced  by  the  same  individual 
under  stress  or  with  an  illness,  etc. 


Warfi2htine  Merit:  Would  permit  the  rapid  receipt  of  supplies. 


Current  Projects: 


Required  Research:  To  uniquely  identify  a  person  so  that  rq)lication  could  be  detected 
and  identification  made  under  different  conditions. 


POC  Mary  Lee  Johnson 
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Spoken  Human-Machine  Dialogue  Workshop 

Panel:  Combat  Service  Support 
(Assistance) 

Title:  NL  assistance  to  medical  doctors 

Concent:  Use  voice  to  assist  the  doctor  either  during  an  operation  or  in  diagnosis  of  an 
illness.  Use  dictation  to  input  to  medical  records. 


Technical  Aiaessment:  Need  expert  knowledge  that  can  be  provided  interactively  with 
the  doctor 


Warfighting  Merit:  Aid  in  medical  diagno»s  of  the  soldier 


Current  Projects: 


Required  Research: 


PCX!)  Miranda  Moore 
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Spoken  Human-Machine  Dialogue  Workshop 


Panel:  Combat  Service  Support 
(Operations) 

Title:  Environmental  warfighting  suit 

Concept:  A  combat  uniform  that  would  consist  of  sensors  and  environmental  awareness 
data  with  voice  query  and  recognition 

Technical  Assessment: 


Warfighting  Merit:  Increased  readiness 


Current  Projects: 


Required  Research: 


POC  Kaiio  Aguilar 


Spoken  Hunian>Machine  Dialogue  Workshop 

Panel:  Combat  Service  Support 
(Operations) 

m  Nuclear,  Biological  and  Chemical  Detection 

Concent:  Develop  a  voice  system  to  add  to  the  field  Chemical  Detect  alarm  system  that 
will  tell  the  “user”  what  agent  was  detected.  Not  just  have  a  beeper. 


Tyfhi^ifal  Assessment:  Will  require  M8  alarm  to  be  able  to  detect  specific  agent. 
Currently  done  manually  by  the  user.  Want  user  to  be  able  to  have  all  inclu^  system. 


Wnrfighting  Merit:  Simplify  soldiers  task 


Current  Projects: 
None 


Required  Research: 


POC  JefifJahnke 
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Spoken  Human-Machine  Dialogue  Workshop 

Panel:  Combat  Service  Support 
(Operations) 

Title:  Voice  activated  radio  frequency  identification  tag 

Concept:  Using  a  tag  that  would  provide  voice  response  to  spoken  queries  concerning 
container  contents  to  include  consignor,  consignee 

Technical  Assessment:  Interface  with  current  and  future  transportation  tracking 
networks 


Warfi£htine  Merit:  Would  provide  Total  Asset  Visibility. 


Current  Protects: 


Required  Research: 


POC  Kario  Aguilar 
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DEPTH  AND  SIMULTANEOUS 

ATTACK 


DEPTH  AND  SIMULTANEOUS  ATTACK 


M-16  MARKSMANSHIP 


Concept: 

Provide  an  automated  spotter  for  marksmanship  training.  Immediate 
voice  feedback.  Error  measurement.  Diagnostic  Andysis  to  shooter. 

Technical  Assessment: 

FY  96  Existing  technology 

Warfi^ting  Merit: 

Reduces  cost.  Improvestraining.  Improves  training  efficiency. 
Required  Research: 


None. 


SHMD  in  Support  of  Forward  Observer 


Concept: 

Observer  gives  calls  for  fire  and  adjustments  orally.  Device 
provides  feedback  of  observer's  command  and  converts  command  to 
digital  data  for  transmission.  Device  is  capable  of  being 
reprogrammed  for  FO's  native  language.  Highly  restrictive 
grammar. 

Technical  Assessment: 

FY96 

Warfighting  Merit: 

More  effiicient  operation  for  FO.  Can  continously  track  target. 
Decreased  sensor  to  shooter  timelines. 

Required  Research: 

Dealing  with  pauses,  starts,  restarts  and  repeats. 
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DEPTH  AND  SIMULTANEOUS  ATTACK 


SHMD  for  the  Conduct  of  Fire  Trainer 


Concept:  The  observer  voices  a  correction  for  artillery  rounds.  The 
system  reads  back  the  correction  and  then  applies  it  back  to  the  fire 
trainer. 


Technical  Assessment: 

FY96 

Warfighting  Merit: 

Realistic  training. 

Cheaper,  better,  and  faster. 
Required  Research: 


None 


SHMD  Support  of  Helicopter  Training 


Concept: 

Student  flies  aviation  simulator.  System  provides  student  instant 
oratory  feedback  of  errors  and  suggested  corrections.  System 
debriefs  student  after  the  mission  and  student  can  query  computer 
orally  for  more  details.  Initially  dialog  fi-om  student  will  be  similar 
to  menu  driven.  Later  phases  will  move  toward  fi’ee  form  text. 

Technical  Assessment: 

Phase  I  -  FY96 

Phase  n  -  TBD 

Warfighting  Merit: 

Instant  feedback  and  no  inhibition  to  query  machine  numerous  time 
vs  querying  instructor. 


Required  Research:  Depends  on  speech  fi'eeform  dialog  requirements. 
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Digitized  Overlays 


Concept: 

Commander  or  staff  officer  views  the  digitized  terrain.  Orally 
request  specialized  overlays  be  displayed  on  the  screen.  Screen 
provides  zoom  capability.  Can  also  select  particular  features  to  be 
displayed  or  erased  from  the  screen.  Artificial  intelligent  agent  will 
compute/  correlate  between  designated  features  on  overlays.  For 
example,  an  enemy  unit  on  the  SIGINT  and  OB  overlay. 
Probability  that  more  than  one  unit  displayed  on  an  overlay  is  only 
one  unit  vs  several.  (Voice  interaction,  touch  screens,  etc.) 

Technical  Assessment: 

Phase  I -FY96 

Phase  n  -  TBD 

Warfighting  Merit: 

Better  picture  of  battlefield,  better  targeting,  cost  reduction,  better 
accuracy,  timeliness,  and  information  profiling. 


Required  Research: 


Force  XXI  Network  Vision  Statement 


Commanders  from  corps  down  must  have  nautural  language  dialog 
access  to  all  pertinent  data  and  information  about  friendly  and  enemy 
forces  on  the  battlefield  and  this  access  is  to  be  supported  by  the 
development  of  intelligent  agents  that  will  searach  for  find  and  presnet  the 
required  knowledge  in  natural  language. 


Research  Issues: 

Natural  Language  Processing 
Common  Sense  Reasoning 
Data  Fusion 

Tasks  related  to  above: 

Spatio-temporal  reasoning 
Deadline  reasoning/planning 
Contradiction-tolerance 
Advice-taking 

Context  sensitivity,  pragmatics 
Coordination/negotiation 
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EARLY  ENTRY,  LETHALITY 
AND  SURVIVABILITY 


MISSION 
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SHMD  Operational 
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SHMD  Assistance 


Tng.  Preparation 


Spoken  Human-Machine  Dialogue  Workshop 


Concepts  Worksheet 

Panel:  Early  Entry  Lethality  and  Survivability 

Training  Assistance  Operations 

(Circle  One) 


Title:  Generalized  multimedia  machine  code  transducer 

Concept:  This  system  receives  multimedia  input  from  user  and  codes  it  for  machine  and 
network  internal  use.  This  system  receives  internal  representations  of  messages  to  be 
delivered  to  the  soldier  and  converts  them  to  multimedia  outputs  for  the  soldier. 

Technical  Assessment:  (Desired  Capabiltities,  Current  Shortfalls) 

This  can  be  done  in  primitive  form  now.  But  huge  improvements  are  needed. 

Warflghting  Merit:  (Value  added  to  Soldier) 

Provides  instanteous  interaction  with  computer  network  for  very  fast  control  of  action  and 
very  fast  return  of  information  to  the  network.  Wide  applicability  to  many  types  of 
fighting  personnel-pilots,  tank  conunanders,  logistics  personnel,  and  commanders. 

Current  Projects:  Current  networking  within  the  aumy. 

Required  Research: 

Multimedia  transducers. 

Multimedia  dialogue  theory. 

Human  factors  work  on  multimedia  systems. 
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LOUISIANA  MANEUVERS 


LOUISIANA  MANETJVERS 


Mitch  Marcus 

Computer  Science  Department 
University  of  Pennsylvania 

Report  of  the  Louisiana  Maneuvers  Task  Force  Working  Group 
This  group  considered  a  range  of  applications  for  speech  technology  to  enhance  warfighting  skills. 
Three  applications  that  were  not  discussed  in  other  groups: 

-The  use  of  speech  recognition  and  synthesis  in  simulated  environments  to  enhance  the  realism  of 
interaction  in  simulated  environments.  It  should  also  be  possible  to  enhance  the  realism  of  battlefield 
confusion  with  a  realistic  replication  of  the  "din  of  battle"  by  synthesizing  parallel  radio  traffic  between 
other  simulation  agents.  Voice  interaction  can  also  be  used  to  greatly  speed  up  the  setup  of 
simulated  scenarios;  CBS  scenarios  can  take  2-6  months  to  develop  currently 

-Speech  recognition  can  play  a  special  role  in  language  training,  if  speech  recognizers  and  synthesizers 
for  the  necessary  languages  can  be  made  available.  The  Army  can  take  advantage  of  advanced 
systems  that  accurately  recognize  and  correct  accent.  We  will  need  systems  that  are  sensitive  to 
dialects;  we  need  Haitian  Creole  as  well  as  Parisian  French. 

-A  voice  interactive  navigation  assistant  for  land  navigation  in  low-light/low-visibility  conditions. 
Map  reading  on  a  ranger  patrol  can  be  lethal;  we  envision  a  combination  of  GPS  and  an  expert  system 
for  route  planning  with  voice  input  and  output.  The  same  kind  of  system  with  a  map  display  will  be 
useful  for  the  Rotor  Craft  Pilot  Associate,  extending  the  current  RCPA  project. 

All  of  these  projects  requires  essentially  the  same  core  language  research,  including; 

-An  understanding  of  situated  discourse  with  mixed  initiative  dialogue  in  constrained  semantic 
domains. 

-Language  processing  of  sentence  fragments  from  casual  speech  produced  under  stress. 

-For  the  navigation  assistant,  speech  recognition  of  whispered  speech. 

-A  codification  of  the  various  Army  jargons,  by  functional  area. 

-Generation  of  spoken  natural  language  from  computer  input. 
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LOUISIANA  MANEUVERS  TASK  FORCE  WORKING  GROUP 


TRAINING 

Title: 

Use  of  speech  recognition  and  synthesis  in  simulated 
environments 

Concept: 

Enhance  realism  of  interaction  in  simulated  environments. 
Enhance  realism  of  battlefield  confusion,  realistic  replication  of 
the  'din  of  battle' 

Technical  Assessment: 

Current  shortfall-current  interaction  is  limited  to  keyboard  and 
mouse,  desire  more  realistic  interaction  to  reduce  differentiation 
between  man  and  machine  speech.  Speech  generation  from 
specific  traffic  to  simulate  the  sense  of  chaos  and  confusion. 

Warfighting  Merit: 

Better  training  makes  the  better  soldier.  Train  as  we  fight  - 
more  natural.  Make  simulation  interaction  more  realistic 

Current  Projects: 

Incorporate  voice  recognition  in  MODSAF 
Required  Research: 

Fundamental:  recognition  of  casual  disfluent  speech  under 
stress  in  semantically  constrained  domains,  must  be  dynamic. 
Simulate  in  radio  and  natural  forms  the  communication  traffic. 
Generation  of  natural  language  from  the  simulation.  Codification 
of  Army  jargon  by  its  several  functional  areas. 
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LOUISIANA  MANEUVERS  TASK  FORCE  WORKING  GROUP 

TRAINING 

Title: 

Voice  Interactive  scenario  generation  and  setup 
Concept: 

The  setup  of  scenario  is  very  time  and  labor  intensive.  For 
example  CBS  scenarios  can  take  2-6  months  to  develop. 

Technical  Assessment: 

Short  term  goal  to  reduce  time  by  one  third  and  long  term  goal  of 
two  thirds  reduction  in  time. 

Warfighting  Merit: 

Make  simulation  more  available  and  reduce  time  to  react  to  short 
term  threat.  Gives  more  time  for  what  If  analysis. 

Current  Projects: 

Required  Research: 

Scenario  generation  jargon;  Grammar 
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LOUISIANA  MANEUVERS  TASK  FORCE  WORKING  GROUP 

TRAINING 

Title: 

Language  Training 
Concept: 

To  make  available  practical  voice  recognition  \  multi-media 
recognition  systems  for  language  training 

Technical  Assessment: 

Speech  recognizers  for  a  variety  of  languages.  In  long-term  ~ 
accent  recognition  and  correction  of  phonemic  features  by 
mapping  back  toward  articulation  (correction  on  pronunciation). 
Must  be  sensitive  to  various  dialects  -  not  just  the  generic\typlcal 
form  e.g.  Parisian  french  and  Haitian  Creole. 

Warfighting  Merit: 

Critical  training  in  Intel  gathering  and  liaison.  Portable  system  fills 
void  in  speaking  skills. 

Current  Projects: 

61  person  lab  at  USMA  with  voice  interactive  multi-media 
stations. 

Required  Research: 

Accent  recognition  and  automatic  recognition  of  accent  and 
dialects 
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LOUISIANA  MANEUVERS  TASK  FORCE  WORKING  GROUP 

ASSISTANCE  APPLICATIONS 

Title: 

Maintenance  assistant 
Concept: 

Hands  free  and  eye  free  access  to  maintenance  system  to 
enhance  field  maintenance  operations. 

Technical  Assessment: 

Should  be  portable  with  spoken  interface.  Speech  input  and 
output  combined  with  current  trouble  shooting.  Advanced 
systems  can  include  3d  rendering. 

Warfighting  Merit: 

New  systems  deployable  with  little  maintenance  training.  Would 
give  users  in  field  immediate  maintenance  capability. 

Current  Projects: 

RTI  project 

Required  Research: 

Same  as  above 
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LOUISIANA  MANEUVERS  TASK  FORCE  WORKING  GROUP 

ASSISTANCE  APPLICATIONS 

Title: 

Voice  interactive  navigation  assistant 
Concept: 

Voice  interactive  land  navigation  in  low-light\low-visibility 
conditions. 

Technical  Assessment\Warfighting  Merit: 

Map  reading  on  a  ranger  patrol  can  be  lethal  ~  combination  of 
GPS  and  an  expert  system  for  root  planning  with  voice  in  and 
output  will  allow  for  safer  more  reliable  navigation  under  low  light 
conditions  and  other  conditions  of  low  visibility.  The  same  kind  of 
systern  with  map  display  will  be  useful  for  the  rotor  craft  pilot 
associate.  RTPA  must  be  able  to  integrate  situational  awareness 
(friendly  and  enemy) 

Current  Projects: 

Extension  of  technology  in  current  RTPA  project. 

Required  Research: 

Navigational  planning;  dynamic  root  planning.  Situated 
discourse,  mixed  initiative  dialogue,  sentence  fragments,  casual 
speech,  whispered  speech. 
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LOUISIANA  MANEUVERS  TASK  FORCE  WORKING  GROUP 
ASSISTANCE  APPLICATIONS 


Voice  interactive  version  of  existing  Trauma 
Concept: 

TRAMAID  expert  system  could  be  enhanced  by  the  addition  of 
speech  InputNoutput,  allows  hands  free  eyes  free 

Technical  Assessment: 

Use  of  current  system  requires  additional  personnel  for  keyboard 
input 

Warfighting  Merit: 

More  efficient  use  of  expert  system  that  will  save  lives 

Current  Projects: 

Required  Research: 

Same  as  above 


LOUISIANA  MANEUVERS  TASK  FORCE  WORKING  GROUP 

OPERATIONS 

Title: 

Enhancement  of  Battlefield  Visualization  systems 
Concept: 

Make  current  designs  more  accessible  to  voice  interaction 
technology 

Technical  Assessment: 

MinimizeVeliminate  use  of  keyboard  and  mouse  to  enable  a  more 
portable  system.  Desired  system  should  be  able  to  clarify  what  Is 
asked  for  as  needed.  For  example,  want  a  multi-media  expert 
which  will  show  unit  status  while  highlighting  recent  changes  in 
speech. 

Warfighting  Merit: 

More  natural  interface,  better  access  to  data. 

Current  Projects: 

Pheonix 

Required  Research: 
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ATTACHMENTS 


Attachment  A 


April  18, 1995 


Amy  Research  Office 
P.O.  Box  12211 

Research  Triangle  Park,  NC  27709-221 1 
E>ear  Sir/Maam: 

You  are  invited  to  attend  a  workshop  that  will  focus  on  spoken  human-machine  dialogue 
(SHMD)  technology  and  systems.  University  research  in  natural-language  processing  is  reaching  a 
level  of  maturity  where  it  is  now  appropriate  to  examine  the  transfer  and  integration  of  natural- 
language  systems  into  requirements  and  prototypes  that  support  the  realization  of  Army  Force 
XXI  capabilities.  There  also  is  a  need  to  identify  and  focus  future  basic  research  to  ensure  that  the 
full  potential  of  natural-language  processing  is  realized  for  both  military  and  civilian  applications. 
The  workshop  will  attempt  to  identify  and  address  the  relevant  issues. 

Specific  objectives  for  the  workshop  include  the  following  (the  Agenda  is  presented  in 
Attachment  A): 

•  Introduce  workshop  participants  to  SHMD 

•  Review  the  status  of  SHMD  technology 

•  Develop  Operational  Capability  Requirements  to  support  TRAD(X!^  BattleLabs  in  Army 
Training,  Assistance  and  Operational  applications  of  SHMD 

•  Identify  basic  research  needs  and  opporturdties  based  on  futuristic  concepts 

This  workshop  is  being  sponsored  by  the  Amy  Research  Office  with  participation  by  TRADOC. 
The  registration  fee  is  $1 10  (see  Attachment  B  for  details)  and  attendance  is  by  invitation  only. 
The  location  and  dates  of  the  workshop  are: 

Sheraton  Imperial  Inn,  Research  Triangle  Park,  NC 

May  30,  May  31  and  June  1, 1995 

Please  direct  any  questions  to  the  following: 

Dave  Hislop:  (919)  541-4255,  hislop<®aro-cmhl. army.mil 

Brad  Shaffer:  (804)  727-4481,  shafferb@monroe-emhl.army.mil 

Your  contribution  will  be  essential  to  the  success  of  this  workshop.  We  look  forward  to  seeing 
you  there. 

Sincerely  yours. 


David  W.  Hislop 
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Attachment  B 


ADMINISTRATIVE  INSTRUCTIONS 
Spoken  Human*Machine  Dialogue  Workshop 

Workshop  POCs: 

Army  Research  Office:  Dr.  Dave  Hislop 

Tel#:  (919)549-4255 

E-mail:  hislop(S)aro-cmhl. army.mil 

TRADOC:  Major  Brad  Shaffer 

Tel#:  (804)  727-4481 

E-mail:  shafferb@monroe-emhl.army.mil 

Workshop  Location:  Sheraton  Imperial  Hotel 

P.O.  Box  13099 

Research  Triangle  Park,  NC  27709 
Phone:  (919)  941-5050 
FAX:  (919)  941-5156 

Hotel  Point  of  Contact:  Ms.  Kara  Cuscaden  (National  Sales  Manager) 

Workshop  Dates/Time:  30 1 300  May  95  to  0 1 1 500  June  95 

Workshop  Attire:  Class  B  uniform/business  attire.  Business  attire  is  suggested  for  all 

participants  at  the  banquet  on  30  May. 

Hotel  Room: 

A  block  of  forty  rooms  have  been  reserved,  under  the  name  Spoken  Human-Machine  Dialogue 
Workshop,  at  the  Sheraton  Imperial  Hotel  at  Government  per  diem  rates.  These  rooms  will  be 
available  on  a  first  come  basis  until  May  5, 1995;  after  which  the  Hotel  will  use  normal 
reservation  procedures  and  charge  normal  prices  for  rooms  that  might  be  available.  The  Sheraton 
Imperial  is  located  2  miles  north-west  of  the  RDU  Airport,  (Exit  Page  Rd.  off  Interstate  40). 

Reservations: 

Participants  are  requested  to  make  individual  room  reservations  directly  with  the  Hotel.  When 
making  reservations,  ask  for  the  block  of  rooms  reserved  for  the  ARO-TRADOC  Spoken  Human- 
Machine  Dialogue  Workshop.  Reservations  must  be  made  before  May  5,  1995  to  receive  the 
Government  per  diem  rate. 
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Transportation; 

Workshop  participants  are  responsible  for  individual  transportation.  The  Raleigh-Durham 
International  Airport  (RDU)  serves  the  Research  Triangle  Park  and  surrounding  region.  The 
Sheraton  Imperial  Hotel  provides  shuttle  transportation  vans  between  the  Hotel  and  the  airport. 
Taxi  is  also  available  for  the  short  five  minute  drive. 

Registration  Fee: 

A  Workshop  registration  fee  is  being  charged  to  each  participanL  This  registration  fee  pays  for 
two  lunches,  the  banquet,  refreshments  during  the  Workshop,  and  other  miscellaneous  expenses 
necessary  for  a  successful  Workshop.  This  registration  fee  is  to  be  paid  to  the  Sheraton  Imperial 
Hotel  and  may  be  added  to  the  hotel  bill  at  check-in. 

Registration  Form; 

To  assist  in  the  planning  for  the  Workshop,  each  participant  is  asked  to  fill  out  and  FAX  the 
registration  form  provided  on  the  following  page  by  May  15,  1995  to: 

Ms.  Ingrid  Agolia 
Tel#:  (919)541-6297 
FAX#:  (919) 541-6965 
E-mail:  iba(a)es.rti.org 
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REGISTRATION  FORM 
Spoken  Language  Dialogue  Workshop 


Name  of  Participant; 
Organization: 


Expectations: 


Each  parucipant  will  be  assigned  to  one  of  seven  Working  Groups  during  breakout  sessions  to  examine  and 
develop  issues  and  applications  of  spoken  language  dialogue.  Each  Working  Group  will  be  chaired  by  a 
representative  for  the  TRADOC  BattleLabs  and  co^haiied  by  a  representative  of  the  Army's  scientific  community. 
Please  indicate  below  your  preferences  (1-7)  for  assignment  to  the  Working  Groups.  Your  resptmse  will  be  used  for 
planning  and  making  Working  Group  assignments;  please  provide  this  information  by  May  15, 1995  to  the 
individual  noted  below. 


Order  of  Woridng  Groups  Prefaence  (1-6) 


Battle  Command 

Mounted  Warfighting  Battlespace 
Combat  Service  Support 
Depth  and  Simultaneous  Attack 
Eariy  Entry  Lethality  and  Survivability 
Louisiana  Maneuvers  Task  Force 


Please  provide  the  above  information  by  May  15, 1995  to: 
Ms.  Ingrid  Agolia 
Tel  #:  (919)  541-6297 
FAX#;  (919) 541-6965 
E-mail:  iba@esjti.org 
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Attachment  C 

Spoken  Human-Machine  Dialogue  Workshop 
Attendees 


Karlo  A.  Aguilar 
U.S.  Army  CASCOM 
ATCL-MT 

Ft  Lee.  VA  23801-600 
(804)  734-2099 
(804)  734-1 174  (fax) 
agui]ark@lee-dnsl  .army.mil 

Alan  W.  Biermann 

Duke  University 

Department  of  Computer  Science 

P.O.  Box  90129 

Durham,  NC  27708 

(919)660-6500 

(919)  660-6519  (fax) 

awb@cs.duke.edu 

Dr.  Jim  Brown 
Research  Triangle  Institute 
3020  Cornwallis  Rd. 

P.O.  Box  12194 

Research  Triangle  Park,  NC  27709 
(919)  541-6975 
(919)  541-6965  (fax) 
jnb@esjti.org 

Tanya  N.  Bui 

U.S.  Army  CASCOM 

ATCL-B 

Ft  Lee.  VA  23801 
(804)  530-1491 
(804)  530-2880  (fax) 
buit@lee-dns  1  .army.mil 

Howard  Burnette 
U.S.  Army  CASCOM 
Attn:  ATCL-MT 
Ft  Lee,  VA  23801-6000 
DSN  687-0586 
(804)  734-0586 


(804)  734-1 174  (fax) 
bumetth@  lee-dns  1  .anny.mil 

Chuck  Campbell 

U.S.  Army  Mounted  Battlespace  BattleLab 

Cdr,  USAARMC,  Attn:  ATZK-MW 

Ft  Knox,  KY  40121 

(502)  624-1963 

(502  624-1932  (fax) 

campbelc@knox.army.mil 

Edward  J.  Carroll 
U.S.  ArmyARDEC 
AMSTA-AR-FSF-R,  Bldg.  95N 
Picatinny  Arsenal,  NJ  07806-5000 
(201)  724-7280 
(201)  724-7279  (fax) 
ecarrol@pica.army.mil 

David  L.  Carter 

Early  Entry  Lethality,  Survivability  BattleLab 

Commander  TRADOC 

Atm:  ATCD-L 

Ft  Monroe,  VA  23651-5000 

(804)  727-4203 

(804)  728-5861  (fax) 

carterd  l@monroe-emh9.army.mil 

Jagdish  Chandra 

Director,  Mathematical  and  Computer 
Sciences 

U.S.  Army  Research  Office 
RTP,  NC  27709 
(919)  549-4254 
(919)  549-4310  (fax) 
chandra@aro-emhl  .army.mil 

Malvin  J.  Chemow 
Director  of  Contracting 
USA  Intel  Ctr./BCBL-H 
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P.O.  Box  748 
Attn:  ATZS-DKO-I 
Ft  Huachuca,  AZ  85613-0748 
(602)  or  (520)  538-0422 
(602)  or  (520)  538-0415  (fax)  533-1600 
(fax) 

chemow  1  %  hua  1  (S)  huachuca- 
emhl  1. army  .mil 

Hugh  Dempsey 

Louisiana  Maneuvers  Task  Force 
Atm:  DACS-LMI 
Ft  Monroe,  VA  23651 
(804)  728-5822 
(804)  728-5810  (fax) 
dempseyh@monroe-emhl  1  .army.mil 

J.  L.  Enriquez 
Department  of  the  Army 
USAADASCH 
Atm:  ATSA-ADL 
Ft  Bliss,  TX  79916-3802 
(915)  568-7613 
(915)  568-2530  (fax) 
enriqueLX®  bliss-emh  1  .army  .nul 

Jerry  Evans 

U.S.  Army  Missile  Command 
Missile  Research,  Development  and 
Engineering 
Center 

AMSMI-RD-AC 

Redstone  Arsenal,  AL  35898-5242 
(205)876-2357 
(205)  876-0640  (fax) 
evans-rd-ac(a)redstone-emh2.army.mil 

Michael  W.  Freeman 
BCBL 

Atm:  ATZL-CDB 
Ft  Leavenworth,  KS  66027 
(913)  684-2357 
(913)  684-2350  (fax) 

£teemanm@  leav-emh.army.mil 


CPT  Mary  Ann  Gilgallon 
CSS  BatUe  Ub,  CASCOM 
Ft  Uc,  VA  23801 
(804)  734-2285 
(804)  734-2280  (fax) 
gilgallm(a)  lee.dnsl.army.mil 

Ed  Greisch 
ACALA-AC-MAEE 
Rock  Island  Arsenal,  IL  61299 
DSN  793-0018 
maeweg(2)ria-emh  1  .army.mil 

Curry  Guinn 

Research  Triangle  Institute 
3040  Cornwallis  Rd. 

Research  Triangle  Park,  NC  27709 
(919)541-8707 
(919)  541-6965  (fax) 
cig(a)es.rti.org 

Carlton  Harris 

HQ  TRADOC,  Battle  Lab  Integration, 
Technology 
and  Concepts  Dir. 

Atm:  ATCD-BP 

Ft  Monroe,  VA  23651 

(804)727-4136 

(804)  727-3445  (fax) 

harrisc@monl-emhl.army.mil 

Robert  F.  Helms 
Research  Triangle  Institute 
3040  Cornwallis  Road 
P.O.  Box  12194 

Research  Triangle  Park,  NC  27709 
(919)  541-7166 
(919)  541-6965  (fax) 
rfh@es.rti.org 

Ray  Higgins 

Aviation  Applied  Technology 
Directorate/Rotorcraft  Pilots  Assoc. 
Bldg.  401 

Atm:  AMSAT-R-TDRP  (R.  Higgins) 
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(703)  883-6435  (fax) 
macmillan(a)ai.mitre.org 

Mitch  Marcus 
University  of  Pennsylvania 
2000  S.  33rd  Sl 
Philadelphia,  PA  19104 
(215)  898-2538 
(215)  898-0587  (fax) 
mitch@cis.upenn.6du 

Wesley  A.  Milks 

U.S.  Army,  Simulation,  Training  & 

Instrumentation  Command  (STRICOM) 

Attn:  AMSTI-EE 

12350  Research  Parkway 

Orlando,  FL  32826-3276 

(407)  381-8789 

(407)  384-2338  (fax) 

milksw@stricom.army.mil 

Walter  Miller 

U.S.  Army  Aviation  Center 
ATZO-CD 

Ft.  Rucker,  AL  36362-5000 
(334)  255-2346/9589 
(334)  255-1214  (fax) 
miller3@rucl.army.mil 

R.  Jorge  Montoya 
Research  Triangle  Institute 
3040  Cornwallis  Road 
P.O.  Box  12194 

Research  Triangle  Park,  NC  27709 
(919)  541-6807 
(919)  541-6965  (fax) 
ijm@es.rti.org 

Miranda  Moore 

U.S.  Army  Logistics  Evaluation  Agency 
LOEA-PL 

New  Cumberland,  PA  17070-5007 
(717)  770-7600 
(717)  770-6702  (fax) 
moore@pentagon-hqdadss.army.mil 


Daniel  B.  Nissman 
Research  Triangle  Institute 
3040  Cornwallis  Road 
P.O.  Box  12194 

Research  Triangle  Park,  NC  27709 
(919)541-5801 
(919)  541-6965  (fax) 
dbn@es.rti.org 

Don  Perlis 

University  of  Maryland,  College  Park 
Computer  Science  Dept,  A.V.  Viliams 
Bldg. 

CoUege  Park,  MD  20742 

(301)  405-2865,  (301)  405-6707  (fax) 

perlis@cs.umd.edu 

MAJ  Brad  Shaffer 

Battle  Labs  Integration,  Technology  and 
Concepts  Directorate 
HQTRADOC 
Attn:  ATCD-B  (Maj  Shaffer) 

Ft  Monroe,  VA  23651-5000 
(804)  727-4481 
(804)  727-2947  (fax) 
shafferb@  monroe-emh6.army.mil 

Dana  Smith 

NAWCTSD 

12350  Research  Parkway 

Orlando,  FL  32826-3224 

(407)  380-4607 

dana@ntsc-rd.navy.mil 

MAJ  Donald  Starck 
U.S.  Army 

ATSF-CBL  (DSA  Battlelab) 

Ft  Sill,  OK  73503-5600 
(405)  442-5647 
(405)  442-5028  (fax) 
starck@sill-battlelab.army.mil 

Bill  Toth 
HQ  CECOM 
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Ft.  Eusus,  VA  23604-5577 
(804)  878-2371 
(804)  878-3108  (fax) 
rhiggins(2)eustis-aatds  1  .aimy.tnil 

David  W.  Hi  slop 

U.S.  Army  Research  Office 

Atm:  AMXRO-MCS  (Dr.  David  W.  Hislop) 

P.O.  Box  12211 

Research  Triangle  Park,  NC  27709 
(919)549-4255 
(919)  549-4288/4310  (fax) 
hislop(a)aro-emh  1  .army.mil 

Melissa  Holland 

Army  Research  Institute 

5001  Eisenhower  Avenue 

Alexandria,  VA  22333-5600 

(703)  274-5569 

(703)  274-3573  (fax) 

holland(3)  alexandria-emh2.army.mil 

Deanna  Jenkins 

Louisiana  Maneuvers  Task  Force 
Atm:  DACS-LM,  Bldg.  83 
Ft  Monroe,  VA  23651 
(804)728-5317 
(804)  728-5826  (fax) 
jenkinsd@monroe-emh  1 1  .army.mil 

Mary  Lee  Johnson 
Data  Systems  Research  and 
Devclopment/Lockheed  Martin 
1099  Commerce  Parks 
M/S  7619,  Rm.  W30 
Oak  Ridge,  TN  37830 
(615)  576-8573 
(615)  574-7624  (fax) 
mlj@oml.gov 

Aravind  K.  Joshi 
University  of  Pennsylvania 
Department  of  Computer  and  Information 
Science 

Room  555,  Moore  School 


PhUadelphia,  PA  19104 
(215)  898-8540 
(215)  898-0587  (fax) 
joshi@lmc.cis.upenn.edu 

LTC  Stephen  A.  LaRocca 

Department  of  Foreign  Languages 

U.S.  Military  Academy 

West  Point  NY  10996 

(914)  938-5286 

(914)  938-3585  (fax) 

gs0416@usma3.usma.edu 

CPT  Kelly  Lauritzen 
Battle  Command  Battle  Lab 
Attn:  ATZL-CDB 
Ft  Leavenworth,  KS  66027 
(913)  684-2362 
(913)  684-2350  (fax) 
lauritzk@  leav-bcbl.army.mil 

John  Lavery 
Army  Research  Office 
Mathematical  and  Computer  Sciences 
Division 
AMXRO-MCS 
P.O.  Box  12211 

Research  Triangle  Park,  NC  27709-221 1 
(919)  549-4253 
(919)  549-4310  (fax) 
lavery@aro-emh  1  .army.mil 

Ming  C.  Lin 

Army  Research  Office 

Math  and  Computer  Science  Division 

Research  Triangle  Park,  NC  27709 

(919)  549-4256 

lin@cs.unc.edu 

T.  Richard  MacMillan 
The  Mi  IRE  Corporation 
M/SZ464 
7525  Colshire  Drive 
MeUan,  VA  22075 
(703)  883-7796 
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AMSEL-RD-C2-BB-CC 
Bldg.  2525 
Ft.  Monmouth,  NJ 
(908)  427-3819 
(908)  427-4087  (fax) 


Qifford  Weinstein 
MIT  Lincoln  Laboratory 
244  Wood  Street 
Uxington,  MA  02173-9108 
(617)  981-7241 
(617)  981-0186  (fax) 
cjw@sstll.mitedu 

James  B.  Widby 
U.S.  Army  BCBL  (G) 

Attn:  ATZH-BLT,  Bldg.  71600 
Ft  Gordon.  GA  30905-5299 
(706)  791-8344 
(706)  791-8346  (fax) 
widbyj(3>bcblg.gordon.army.niil 

David  ^^lliamson 
USAF,  WL/nGP-2 
2210  Eighth  St,  Ste  1 
WPAFB,  OH  45433-7511 
(513)  255-6696 
(513)  476-4547  (fax) 
williadt@wl.wpafb.af.mil 


Attachment  D 


Time 

1300-1315 

1315-1415 

1415-1515 

1515-1530 

1530-1615 

1615-1745 


1745-1830 

1830-1900 

1900-1945 

1945-2015 


Agenda 

Spoken  Human-Machine  Dialogue  Workshop 

Tuesday,  May  30, 1995 

_ _ Activity _ 

Welcome/Introduction 
Technology  Status/Overview 
Human-Machine  Interactions  in  2020 
Natural  Language  Processing  Overview 
Break 

Spoken  Language  Dialogue 
Demonstration 

Spoken  Human-Machine  Dialogue 
Maintenance  Trainer  and  Intelligent  Assistant 

Open  Time/Social  Hour/Dinner 

Open  Time 

No  Host  Social  Hour 

Dinner 

Dinner  Speaker 


Responsibility 
J.  Chandra 

C  Weinstein 
M.  Markus 

A.  W.  Biermann 

J.  N.  Brown 


T.  Edwards 


0800-0850 

0900-1100 

1100-1130 

1130-1230 

1230-1330 

1330-1430 

1430-1500 

1500-1700 

1700-1730 


Wednesday,  May  31, 1995 

Examples  of  Army  Natural  Language  Applications  CTT  K.  Lauritzen 
Working  Groups  Exan^ation:  Training  Applications  Dir,  BattleLabs 

Break 

Working  Groups  Examinations:  Assistance  Applications  Dir,  BattleLabs 

Lunch 

Working  Groups  Examinations:  Assistance  Applications  (Cont)  Dir,  BattleLabs 

Break 

Working  Groups  Examinations:  Operational  Applications  Dir,  BattleLabs 
Summary  of  Day's  Activities  J.  Chandra 
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Time 


0800-0915 

0915-0930 

0930-1045 

1045-1200 

1200-1300 

1300-1430 

1430-1500 

1500 


_ Activity _ 

Thursday,  June  1, 1995 

Working  Group  Repon  Out;  Training  Applications 
Break 

Working  groups  Report  Out:  Assistance  Applications 
Working  Group  Report  Out;  Operational  Applications 

Lunch 

Presentation  of  Specific  Projects 
Workshop  Closing  Remarks 
Workshop  Conclusion 


Responsibility 

Dir,  BattleLabs 

Dir,  BattleLabs 
Dir,  BattleLabs 

Dir,  BattleLabs 
J.  Chandra 
J.  Chandra 
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